7 Learnings in the Journey from Prototype to ProductPosted on by Rik Higham
Skyscanner’s Car Hire metasearch product recently passed a significant milestone: x10 growth within two years. Specifically, we are bringing in 10 times as much monthly revenue as we were two years ago. In July we even generated more revenue than our Hotel meta-search product, which is a much more lucrative industry.
In the journey to achieving this, from prototype to a fully-fledged product, the team I was part of learned a huge amount. We believe these learnings can be applied to the introduction of any new product, whether internally here at Skyscanner or not.
From the power of the A/B test to the need to be an evangelist: here are the seven key learnings that really stood out for me.
1. Learn from Users
It sounds obvious but I don’t think its importance can be overstated. The biggest insights we had came from watching people hire cars and speaking to them about it, coupled with analysing the behaviour of people using our product. Understanding what’s important to people and reading between the lines to discover their pain points allowed us to develop unique solutions that addressed their needs.
Once you understand the problem it’s unlikely your first attempt to solve it will be a runaway success. Prototyping is a great way to quickly and cheaply test an idea. Will people find it valuable? Will they find it easy to use? Even something as simple as drawing an interface on sheets of paper can yield remarkable insights. As you gain more confidence and hone in on the important aspects higher fidelity models can be created and tested. Naturally you need to caveat this by the number of people you test with and their background.
3. A/B Test
A/B testing is an excellent technique for measuring whether your feature is valuable and usable. The foundation for a meaningful A/B test is a solid hypothesis. One that is based on a quantitative or qualitative insight from which we can form a prediction that a specific change will cause a measurable impact. When it comes to analysing results the important part is to assume your change had no effect, or as they often say in scientific research: prove yourself wrong. Confirmation bias is an easy trap to fall into (read more: Design like you’re right / Test like you’re wrong)
4. Work with Your Partners
The people who use our product are only half the picture. While the Product Manager should be the heart and voice of users it’s easy to assume that ‘users’ means the people searching for hire cars (in our product). However, the deals are only there because hire companies also get value out of working with us. When we made a significant improvement to our interface we changed the way people used it. Whilst more people were booking car hire, we had to work with our partners to help them understand the change and in some cases modify our commercial agreements to reflect this new behaviour. More recently, we have moved towards product comparison rather than solely price comparison. This allows our partners to highlight what’s included with each deal and give more context around the different prices.
5. Product Discovery (and MVPs) are Hard
The Car Hire team had an innovative 2015 but the one that stands out to me is airport transfers (taxis/shuttles/etc to and from the airport). This is a growing industry and a number of brokers and travel agents are entering in to it. So we knew there was a market but we wanted to test whether it was an opportunity for Skyscanner or not.
We added banners to various places on our website that highlighted the value in booking a transfer and linked to an external provider. We tried different messaging, different partners, Skyscanner-branded and partner-branded booking sites. In each case we got a large click through rate but very few bookings. It wasn’t a successful test but it didn’t feel clean cut enough to say there was no opportunity.
I hypothesised that people were expecting a Skyscanner comparison site, not a single partner. So we took a punt and built as small a version as we could. Despite cutting a number of corners (eg. no unit tests, to the horror of our Q&A engineers!) it still took a lot of effort.
However, the comparison funnel performed much better than a simple link to another website. We have since made it more robust, added more partners, and launched in more markets. The world’s first airport transfers meta-search product.
6. Working with Third Parties can be Tricky
Our Car Hire app was previously developed by a third party company. Their engineers were excellent, but working with a remote external team came with challenges. All our meetings were held over Skype, and a number of things were lost in translation. It was also hard to be agile. Communication over email was slow and not as effective as being in the same room and it was easy to make incorrect assumptions (on both our parts). As a result, we gravitated to more waterfall-like processes. Whenever the team visited our Edinburgh office we achieved more in those few days than we did in weeks normally. In retrospect, we should have done more office visits. Another change that made a huge improvement was using Slack, which enabled much more frequent communication and quick-fire questions and answers.
7. Evangelise Your Product
As the smallest product, with the smallest team, we were often not a priority. However, car hire is a huge, global industry that complements flights well so there was a clear opportunity. A large part of my role became internal PR. I had to work hard to sell the vision and opportunity. To make people care about our product. Part of this was pitching internally for resources (while setting clear expectations). Part of it was demonstrating progress with hard facts and KPIs.
After two years guiding the Car Hire team, it was a bittersweet moment when I realised it was time to move on. Time for me personally to step outside my comfort zone and accept a new challenge within Skyscanner. Time also to let someone with a fresh pair of eyes take a look, challenge things that had become accepted ‘facts’, and explore new avenues.
Rik is a Senior Product Manager at Skyscanner, working on Discovery and Inspiration