Return to Articles 5 mins read

SEO A/B Testing and Speed: How and why it matters

Posted June 6, 2022 by Will Critchlow
Last updated August 7, 2023

The most common challenges we hear of with SEO testing relate to getting the foundational tools and testing methodology in place, and being able to run sensitive enough statistical analyses. Once teams have the basic ability in place to run tests, however, the biggest differentiator of successful commercial outcomes is testing speed and cadence. The way to get ahead of competitors is to run more tests per year than they do.

The way to get ahead of competitors is to run more tests per year than they do

SEO A/B testing is a data-driven approach that aims to prove the value of SEO. There are several reasons our customers choose SearchPilot as their solution to effective SEO A/B testing. Many of those reasons relate to factors that help speed up a team’s testing rhythm: the low lift required of our customers' internal teams, automation of test configuration, automatic test analyses, as well as the iterative testing process that our platform enables. With SearchPilot, SEO A/B testing is robust and teams are able to achieve winning SEO results faster and at scale.

Here’s an example of an update from one of our large customers talking about their testing cadence and the winning results they have got as a result:

27 tests run between February and June: 9 positive, 5 with double digit growth percentages

Keeping up with Google and staying ahead of the curve

The SEO industry is always changing. Google does a great job of keeping us on our toes as we constantly update our sites and do our best to provide quality content that meets users’ needs and provide Google with sufficient information. Not only do sites need to keep up with Google but with their competitors as well. While SEOs tend to exchange ideas and information on best practices to rank higher, what we’ve found is that best practice doesn’t always yield increased organic traffic. Between Google updates and competitor activity, it can be challenging to stay ahead of the curve and have your SEO efforts come out on top. The best way to be sure of this is to perform SEO A/B testing, and not only just to test, but to test effectively and quickly.

Low lift on internal team

The placement of SearchPilot within a business’ technology stack enables immediate changes to a site, managed by the SEO team. The platform has the power to make any change to a website’s HTML and does not rely on client-side JavaScript to do so.

How SearchPilot is deployed

That means the team no longer has to go through long development queues in order to implement an SEO-related change or be constrained to the limited number of edits that may be allowed to them via their CMS. Once we find a winning test, we can globally implement the change for an immediate and swift rollout to maximise the win. The biggest benefit of this approach is that engineering and product teams only need to work on changes that have a proven business benefit, freeing up their time for other improvements and improving the working relationship between product and SEO.

Automation of test setup

A key factor in the pace of SEO testing is the time it takes to set up each test. SearchPilot enables easy segmentation of a website into what we call site sections - generally these are pages that use the same template, but we can also split the same page type into subsections to enable multiple tests to run at once. During a test a site section is then split into control and variant groups using our algorithmic bucketing engine.

Automatic page bucketing using salt shaking

Not only does this remove a manual step, but it also enables higher quality control/variant groups through a process we call “salt shaking” which builds groups of statistically similar pages. For tests with a predefined variant group (e.g. where new content is being written for the test pages), SearchPilot can build a suitable control group of pages that are statistically similar to the chosen variant pages.

From our extensive testing experience at SearchPilot, we know the same test does not always yield the same results in different markets or even on a different site within the same industry. Because of that, we encourage our customers to perform the same test across different page templates and even domains (where it makes sense). The platform allows for test builds to be “reused” which makes test setup across similar templates almost effortless. Teams no longer have to start from scratch to build the same test, and this saves time for all parties involved.

Automation of test analysis

Our tests account for any external factors that could potentially influence a manual analysis, such as Google algorithm updates, seasonality, competitor actions, etc. With SearchPilot, SEO teams can be confident that (given proper test setup) the results will be controlled for external effects that could have skewed the test data.

Once a test is pushed live, the platform performs the analysis right in the dashboard, making complex and often time-consuming data downloads from analytics and Search Console unnecessary. The platform’s state of the art neural network carries out the statistical computations for you daily.

Accuracy of SearchPilot neural network model

Additionally, the SearchPilot professional services team has advanced tools that they leverage to analyse complex tests, remove outliers, and segment the data without having to spend an unnecessary amount of time in spreadsheets.

The test analysis provided by the platform in conjunction with additional analysis done by the professional services team, allows businesses to iterate quickly and efficiently. By getting quick answers to test status and outcomes, teams can move onto the next test as quickly as possible.

Testing and re-testing

When you put all these variables together - low lift on internal teams, automation of test configuration, automation of test analysis - the ability to test and re-test is simply much faster with SearchPilot in place.

In addition, for any winning SEO test, deploying the changes is fast and efficient within the platform. Without much follow up work to do after a winning test has been rolled out, teams are empowered to iterate once again and get another test up and running soon after.

Wrap Up

For teams looking to move forward with a data-driven approach to SEO and who want to do so in a fast and efficient manner, SearchPilot may be the solution. To learn more about the features and functionality of the SearchPilot platform, feel free to reach out for more information or a demo with our team.

Image credit: Felix Mittermeier.