Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
Across most verticals, websites commonly will have text below the fold that predominantly exists to target search engines, not users. We’ve all seen it and we all know hardly anybody reads it other than Googlebot.
Unsurprisingly, John Mueller has previously advised against including this kind of text. The comment, shared in a webmaster hangout, was in reference to e-commerce websites, but it applies more broadly:
Many e-commerce websites optimize their categories by adding a big chunk of text under the product listings. Nothing except an h1 heading above the fold. I don’t consider this good usability since users need to scroll all the way down to read this. Does Google treat this content the same as any other or would you for improving rankings recommend putting the category text above the fold?
From our point of view, that’s essentially keyword stuffing. So that’s something…which I would try to avoid.
I’d try to stick to really informative content and put that in place where you think that users will be able to see it. Especially if it is content that you want to provide for users. And more than that I would think about what you can do to make those pages rank well without having to put a giant paragraph of content below the page.
John Mueller, Google
It’s common for Google’s best practices to come into conflict with what actually works in the real world, so we wanted to put this to the test.
We took a client in the travel sector, whose key transactional pages had copy that appeared to be of relatively little value to users and tested removing that copy. Our client had hired freelancers to write this copy, so they also wanted to know if hiring freelancers was a worthwhile investment to inform future business decisions. What were the results?
Don’t go and start removing all your strictly SEO targeted text without a plan to enrich those pages, because removing this text resulted in a 3.8% drop in organic sessions for our client, and an estimated loss of 9,800 organic sessions a month!
Here’s what the resulting graph looked like:
What do we take away from this result? Is John Mueller lying to us? Not exactly. This test does confirm that the text our client had below the fold on these pages was contributing to search performance. A couple of things we want to bear in mind though:
- This content was written by freelancers, so it may meet John Mueller’s recommendation to stick to content that is informative. The part we can call out though is that this text is not located in a place where users are able to see it without scrolling
- Related to that point, we still haven’t determined if our client’s specific text was adding value to the page, or if any relevant SEO content will benefit rankings. This same client also has similar text on other pages that have been auto-generated, we’re hoping to test removing that next
So, should you consider keeping relevant content on your key pages even if it’s below the fold? As with a lot of tests that we’ve been running, there are no hard and fast rules.
Our advice would be to test it and monitor the impact on your own landing pages. At SearchLove London and San Diego and a recent webinar, Dominic Woodman discussed how in many cases we were seeing varying results for different clients across different verticals (see his full SearchLove session here). As always in SEO there isn’t a silver bullet.
Please feel free to get in touch if you want to learn more about this test or about our SEO split testing platform more generally either via email or come and say hi on Twitter.
How our SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.