In this week’s #SPQuiz, we asked our followers on LinkedIn and X (formerly Twitter) followers if AI generated content would help pages earn more organic traffic (the added content was labelled as AI-generated).
Here’s what they thought might happen:
Poll Results
The majority of our followers believed that this additional content would result in an inconclusive result.
The Case Study
In 2023, the SearchPilot team witnessed the results of our first ‘AI’ generated content experiment. This was just a few months after the public launch of ChatGPT 3.5 in November 2022. It feels like just yesterday… By today’s standards, ChatGPT 3.5 is limited, and by tomorrow, it may feel almost infantile.
In the background, we’ve been testing AI and AI-enabled tools for use cases in SEO experiments focused primarily around text content generation.
This case study is similar to that previous AI case study: an ecommerce customer wanted to leverage AI to see if it could write supplemental product information for their product detail pages. A product description already existed on their pages, so that was fed as an input to the model for context. The black box was instructed to generate a three to five sentence paragraph that spoke more to the ‘Product highlights’ than a description of the product itself.
Once the content was generated, SearchPilot added this content to their product detail pages. For reasons both moral and legal, we appended ‘**This content was generated by AI’ to the end of the content.
What Was Changed
A 'product highlights' container was added to product detail pages (below the fold) with a disclaimer at the end stating the content was generated by AI.
Results
Before we get into analysis, you may notice this chart is looking a bit different from our typical case study results pages. This is an update we’ve made on our side to better communicate the results of the test end state, rather than focusing on the path to the end result. It’s a clearer way to summarize the test’s final outcome, rather than its progression over time.
Nearly two years after our first AI content test. Is this really surprising? Stripping out AI for a moment, we’re adding more personalized content to the page and bulking things up. Following an ages-old SEO fundamental still seems to provide benefit to organic traffic and search ranking competitiveness.
One of the more interesting takeaways we pulled from this experiment is that the inclusion of ‘**This content was generated by AI’ doesn’t seem to bear any weight into how Google Search ranks the page or values the content.
With all of our tests, the context of the page, the brand, the industry competitiveness, the page’s pre-existing ability to rank, and additional factors can all play into the results of an experiment. We were excited to find another winning test for our customer and we moved forward with deploying this change out globally through the SearchPilot platform.
Have you been experimenting with AI tools for developing website content? Take the guess work out of “Is AI content working for us” and take control with SEO A/B experiments of your own with SearchPilot!
To receive more insights from our testing, sign up for our case study mailing list, and please feel free to get in touch if you want to learn more about this test or our split testing platform more generally.
