Split-Testing for SEO purposes has some similarities with traditional CRO focused split-testing, and some big differences. The primary difference, which can be difficult to grasp initially, is that CRO testing buckets users into two groups, whereas SEO testing buckets pages into two groups.
Test Setup
With CRO focused testing, user A and user B may see different versions of the same page to one another. For example, user A may see a tabbed layout (the control) and user B may see a flat layout (the variant) of a product page; if user A refreshes the page they will continue to see the tabbed layout (as they have been bucketed as control).
With SEO focused testing, user A and user B will see the same version of any page as one another, as will search bots and any other visitors. However, different pages in the same family of pages (e.g. product pages, article pages, or region pages) will appear differently; for example the blue widget page may have a tabbed layout and the green widget page a flat layout. These pages use identical templates and similar content to one another. It is important to recognise that there still only exists one copy of each page - it will either be a control or a variant, but not both.
In this fashion a group of pages, such as product pages, will be split into two variations which will have some change that we are looking to test. Search bots, including Googlebot, will crawl the pages and see the same thing as users would see for each page.
Measuring Impact
There will now be two types of pages in the search index, and each type of page may differ in terms of organic search performance. The variant group of pages now may begin to perform better than the control group of pages, which would result in (relatively) more organic search traffic to those pages.
The SearchPilot platform connects to your analytics via an API and records the organic search traffic to both the control and variant pages. It can then do an analysis of which group of pages are performing better and show you the expected impact were you to roll that change out across all of your pages.
The mathematics of the analysis system account for other factors that may change. Things such as:
Google Algorithm Updates
Seasonality
Competitor’s changes
Unrelated changes to the site
Are all controlled for by the platform, using the control pages to change the expected ‘baseline’ behaviour for the variant pages.
