Return to Latest SEO test results 2 mins read

Can third party user ratings increase organic traffic?

Posted on January 27, 2022 by Sam Nemzer

Start here: how our SEO split tests work

If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.

In this week’s #SPQuiz, we were tackling the topic of star ratings ⭐️⭐️⭐️⭐️⭐️. We asked our followers whether pulling in the average rating from a third-party review site would have a beneficial impact on SEO performance, in the knowledge that rating rich snippets were being attained in search results. Here’s what they thought:

Twitter poll showing 80% of votes for positive, 20% for no detectable impact, 0% for negative

The vast majority of the people who voted thought that this test would have the expected result - a positive impact on organic traffic, and nobody thought it would harm performance. In fact, the 20% who said the test wouldn’t have a detectable impact were right!

The Case Study

User reviews can be a great enhancement for your website’s SEO performance for a few reasons: they add relevant user-generated content to the page, they can act as trust signals for users, and (with the necessary schema) star ratings can be pulled into search results to enhance your snippets. 

That being said, it can be a high effort exercise for a business to generate their own review content. Users have so many platforms where they post reviews - Google and Yelp are two big ones - and it can feel like the reviews that belong to one site could be spread across the whole internet.

One of SearchPilot’s customers, a business with hundreds of physical locations, at the time of the test didn’t have any first-party reviews. In the meantime, they wanted to test the SEO value of adding star ratings to their pages. To do this, they added content on the landing pages of each of their branches, showing the average rating that location had garnered on a large third-party review site. The content included a link to the location’s page on the third-party review site . Additionally, they marked up the pages with aggregateRating schema JSON-LD markup.

Mockup of landing page with and without a block of content showing the aggregate rating from a third party site

A few days after launching the test, the markup was having the intended impact - star ratings started appearing in search results. Using third-party data for ratings isn’t something we had previously tested on any other site - we had only tested marking up first-party ratings - so we were also surprised to see the ratings come through in the search results.

Control
Variant

Despite the star ratings appearing in search results, the test did not have a statistically significant impact on the pages’ organic search traffic after running for around seven weeks.

Chart showing no statistically significant impact from the test

We can interpret this as meaning that the addition of star ratings to search results didn’t strongly encourage users to click through to the customer’s site. This was a surprising result, and has not deterred the customer from pressing forward with gathering their own first party reviews, as it is likely that the additional content of the reviews on the landing pages will have an impact on top of the star ratings alone. They are also investigating the conversion rate impact of having more trust signals in the form of ratings on the landing pages.

How our SEO split tests work

The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:

  • By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
  • The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
  • We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).

Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.