How running 1,000+ SEO A/B tests delivered 27% organic growth
Skyscanner is a global leader in travel, with more than 100 million people worldwide using its app and website every month.
By the end of 2020, with lockdowns in effect worldwide due to COVID-19, Skyscanner’s organic traffic was down 35 percent year-on-year. As travel restrictions began to lift, Skyscanner knew that restoring revenue meant getting customers back on its website. But they had an even bolder aim. They didn’t just want to return to pre-pandemic traffic levels — they wanted to exceed them.
Download the full
customer success story
Our key discovery
When organic traffic falls sharply, the pressure is to ship changes fast and hope they work. Skyscanner showed that testing every change first is what gets you back on top, and past where you started.
The same change doesn't land the same way in every market. Without testing, you only find out after the damage is done. With it, you keep the wins and avoid the losses.
One test paid for the program
Skyscanner averaged 28 positive tests a year with SearchPilot. With the business benefits proven in advance, they could focus on changes that impacted their bottom line. For example, a single test in 2022 delivered traffic uplifts across five markets, and the revenue it generated covered the full cost of SearchPilot for the year.
Costly mistakes avoided
Not all tests have positive outcomes, but negative results are just as valuable. One test indicated a proposed change would lead to a two percent drop in traffic in the UK and a 17 percent drop in France. Had Skyscanner implemented this change, it would have cost tens of thousands of pounds per year for those two domains alone.
The challenge
With travel demand well below its pre-pandemic peak, driving up search visibility was key. The Skyscanner team needed an aggressive SEO strategy to improve their rankings and secure a larger share of organic traffic. They doubled down on their marketing efforts, but recovery was slow. By 2021, Skyscanner’s organic traffic was still down 30 percent on 2019 levels.
But with a substantial global web presence across many domains, they faced unique challenges. Site-wide changes were time-consuming, resource-intensive, and posed significant risks. If the changes had a negative impact, the consequences could be devastating.
To identify the most impactful changes and avoid costly errors, they needed a quick and risk-free way to test their SEO initiatives across multiple domains. SearchPilot was on hand to help deliver an aggressive SEO testing strategy — with award-winning results.
Our solutions
A focus on the changes that matter
Thanks to SearchPilot’s SEO A/B testing platform, Skyscanner’s SEO team could test any proposed changes and provide proven impact numbers before making site-wide updates. They were able to measure and share their success with the wider organization, including senior leadership, making sure they received the credit they deserved.
An advanced neural network model
SearchPilot’s neural network model used over 46 million data points to support Skyscanner’s tests, alongside the advanced analyses of SearchPilot’s professional services team.
This approach was crucial to handling the noise in Skyscanner’s data caused by changing travel restrictions.
A global testing program
Deploying SearchPilot to their 20 most strategically important domains allowed Skyscanner’s SEO team to test experiments across multiple markets. They could then implement changes to the global template if it proved positive in some markets, so long as it showed no harm in others.
Features for scalability
SearchPilot offered a range of features to support greater scalability, including the ability to copy and paste changes across domains, autonomous bucketing and modelling, and support from an experienced professional services team.
As a result, Skyscanner delivered a large-scale testing program while maximizing the impact of their internal resources.
Custom project management
Using an API endpoint built by SearchPilot’s engineers, the Skyscanner SEO team created a custom tracker in Google Sheets that logged all planned, running, recently ended, and completed tests. The API endpoint meant this all updated automatically, so the team had on-demand access to the latest data.
Ability to build complex tests
Nearly every type of website change can be made through SearchPilot’s server-side platform, opening up a huge variety of A/B testing options. This means Skyscanner could test anything from changing title tags and implementing schema all the way up to full template redesigns.
Award-winning program
Our collaboration with Skyscanner, which included rolling out a global testing program of 1,100 tests in three years, even won Best Use of Data (SEO – large enterprise) award at the 2023 European Search Awards.
