Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled AI discovery and SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
Heading hierarchy is one of those SEO fundamentals that seems straightforward in theory but can be surprisingly nuanced in practice. The result: on the same retail site, moving product names from H2 to H4 tags increased organic traffic by 14% on brand pages, but the exact same change dropped traffic by 8.5% on product listing pages.
What We A/B Tested
Listing pages often show many products at once, and on many sites every product name is wrapped in an H2. That can leave a single page with a long list of headings that all claim the same level of importance.
Heading tags help search engines understand a page's structure and identify its most important content. An H1 typically describes the main topic, H2s introduce major sections, and H3 to H6 tags cover the smaller details within them.
On a listing page, the collection itself is the main topic, not any individual product. One of our retail customers believed that placing every product name in an H2 gave Google an inaccurate picture of the page, making each product seem as important as the main sections. By moving product names to H4s, we expected Google to better understand the page's structure and give more weight to the keywords in the remaining H1, H2, and H3 tags.
What we changed
We first tested this on the customer's brand pages, which show the full range of products from a single brand. We changed the product name headings from H2 to H4, while the product names, page content, layout, and all other headings stayed the same.
.png?width=750&height=393&name=image%20(6).png)
Results on brand pages

The test was positive at the 95% credible interval, with a +14% increase in organic traffic. On a brand page, the brand name is the main topic, and the product names mostly repeat it. Having many of them in H2s may have competed with the page's main heading, and moving them to H4s could have given Google a cleaner signal about what the page is about.
Testing the same change on PLPs
With the positive results from the first test on brand pages, we ran the same experiment on a different set of listing pages on the same site, expecting the same outcome.
This time, the result was negative at the 95% credible interval, with an -8.5% decrease in organic traffic.
A category page can rank for a broad range of related searches, and product names often account for much of that keyword variety, for example, specific product types, sizes, or features. Reducing the weight of those names may have made the page seem less relevant to those searches, which could explain the traffic drop.
Heading changes don't carry over between page types
Because both tests ran on the same site with the same change, the page type is the clearest difference between them. The outcome wasn't the heading level itself, but what the product names contributed to each page.
On brand pages, the product names mostly repeat the brand's name, so giving them less weight helped the page focus on its main topic. On PLPs, product names carry many of the keywords the page ranks for, so giving them less weight made the page less relevant to those searches.
Before changing heading levels across a site, check which queries each page type ranks for and whether the content you plan to demote helps you win them. A win on one template isn't proof the change will work on another, so test each page type on its own.
To receive more insights from our testing, sign up for our case study mailing list, and please feel free to get in touch if you want to learn more about this test or our split testing platform more generally.
How our AI discovery and SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a credible interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how AI discovery and SEO testing works or get a demo of the SearchPilot platform.