Thoughts from the SearchPilot team

Repeat founders tend to put deliberate effort into defining their second companies’ mission and vision much earlier than they did the first time around. I noticed this trend a few years ago while struggling with mission and vision work at Distilled. Not only do repeat founders typically work to make sure that everyone on the team is able to repeat a crisp mission from very early on, but they are often deliberate in defining the core values while the team is still very small.

Running A/B experiments for SEO, as opposed to for users, is a complicated subject. There’s a lot of confusion over things like test design, controlling for external factors and some of the tools that can help you run experiments. This is a guide that covers all of that and more along with case studies and example tests. There’s a lot to cover, so feel free to jump ahead to any of the sections that interest you the most:

It has been hard to miss the buzz around the latest natural language generation (NLG) capabilities recently, as the OpenAI group released machine learning models trained on ever-larger amounts of written content. This has culminated, at the time of writing, in the beta release of an API to GPT-3. In my opinion, the latest ML models can now create useful content that is worth testing for SEO and user benefits.

Just over four years ago, we broke the news that we were working on the early stages of what would become SearchPilot. Back then we wanted to help clients get changes implemented faster by separating marketing / SEO-driven changes from the engineering backlog. Not only that, we wanted to help businesses understand the impact individual changes were having on SEO traffic and the bottom line. Since then we’ve run tests on all kinds of sites across different industries and sectors, and processed millions of dollars of online revenue for some of the world’s biggest sites.

If you asked a UX professional whether adding upsell items to a checkout process results in more sales or more abandoned carts, they’d tell you to test it — second guessing users is seen as foolish. Yet for many companies, SEO has no testing at all, just endless reams of best practice and hand waving. Our senior consultant, Dominic Woodman, had the opportunity to change role for a year and got the chance to treat SEO differently, running over 50 SEO A/B tests across a range of different websites.

Across most verticals, websites commonly will have text below the fold that predominantly exists to target search engines, not users. We’ve all seen it and we all know hardly anybody reads it other than Googlebot. Unsurprisingly, John Mueller has previously advised against including this kind of text. The comment, shared in a webmaster hangout, was in reference to e-commerce websites, but it applies more broadly: Many e-commerce websites optimize their categories by adding a big chunk of text under the product listings.

As much as we raise alarm about the constant change in Google SERPs, us SEOs love new things with FAQ schema being the latest example. Some are reporting how much they love it, others are saying they’re losing clicks because of it. Here’s a handful of the Twitter buzz: Does winning FAQ snippets improve click-through rates? Naturally, we wanted to test it using SearchPilot to find out for ourselves. In the vertical we have been testing in, commercial pages winning FAQ snippets have quickly started to dominate the landscape.

One of the important features of SearchPilot is taking care of the statistics and measurement automatically. Any time you set up a test, the platform will automatically: Create a forecast of expected traffic to the group of pages you’re testing on Create statistically similar buckets of control and variant pages Measure the organic traffic to the two buckets Calculate whether any difference in traffic is statistically significant When we first built the tool, we used a measurement model that was based on the paper Inferring Causal Impact Using Bayesian Structural Time Series, which was published by Google.

)-- The SearchPilot platform is most famous for SEO A/B testing and more recently full-funnel testing. But fewer people are familiar with one of the other main features; the ability to act as a meta-CMS and change pretty much anything you want in the HTML of your site, without help from your development team or writing tickets. SearchPilot is platform independent, sitting between your website servers and website visitors, similar to a Content Delivery Network (CDN), as shown in the below diagram.

In How Split Testing Is Changing Consulting, Will sums up why high priority SEO changes linger in developer backlogs, and how we’re addressing these issues with our SearchPilot platform that allows us to test and roll out these recommendations without using our clients’ developer resources: we can substantiate best practices like H1 changes, alterations to internal links, and rendering content with and without Javascript. Let’s get started with three tests you should try to see if you can increase organic traffic to your site.