Winning AI citations isn't the same as winning in search | With Cyrus Shepard

To me, Cyrus Shepard is the single best hype man in SEO.

His MozCon MC intros were always incredible, and whenever he shares SearchPilot's work on social media, I know he is probably going to get more attention for it than I will. So I wanted to start this conversation by trying to return the favour.

Because Cyrus has done the real work.

He worked as a Google Quality Rater so he could report back from first-hand experience. His research has looked at 81,000 title tags and 23 million internal links. More recently, he downloaded every AI citation experiment, study, explainer and patent he could find, narrowed them down to the 54 he thought were most useful, and read every one so the rest of us could benefit.

That was the starting point for this conversation: what does the evidence actually tell us about AI citations, click signals, content quality and the increasingly messy relationship between Google and AI discovery?

At SearchPilot, we have already seen something awkward in real GEO experiments. A single change can improve LLM discovery while making Google performance worse. Cyrus's research gets at the same problem from another direction. Citations can be surprisingly easy to win. The harder question is whether winning them actually helps the business.

˙✧˖ AI-written summary

Below is an AI-assisted summary of the webinar conversation. This is not a word-for-word transcript but is included to help you find the key parts of the conversation.

Research should give teams test ideas

Will framed one of the biggest benefits of Cyrus's work in a very SearchPilot way: good research gives teams better hypotheses.

That has been true of Cyrus's previous studies. His research into Google's title rewrites gives teams ideas about what to test in titles. His analysis of 23 million internal links gives teams ideas about linking structures. His newer work on content effort, click behaviour and AI citations does the same thing for the next generation of experiments.

The point is not to take a research finding and turn it immediately into a new "best practice". It is to turn it into stronger hypotheses.

That distinction matters because even well-supported industry findings can behave differently on a particular site. Research tells teams where to look. Testing tells them whether the idea works for their own pages, users and business.

Cyrus's test ideas start with the user

Will asked Cyrus what he still wishes somebody would test after reading all of this research.

One idea was really simple: answer the main question of the page in the first sentence rather than making users work their way through an introduction. Another was testing AI-generated summaries near the top of articles. Cyrus also wondered whether showing the negatives of a product, destination or recommendation could make content more credible.

That last idea partly comes from Cyrus's experience as a Google Quality Rater. A page saying everything is perfect does not always feel trustworthy. People behave similarly with reviews. A product with nothing but glowing feedback can make shoppers suspicious, while an honest explanation of the drawbacks can make a recommendation feel more believable.

Will pointed out that CRO teams have seen similar effects. On highly rated products, surfacing negative reviews can help because shoppers are looking for the downside anyway. Whether that same behaviour affects organic search is exactly the sort of thing worth testing.

Content effort is harder to fake

The conversation then moved into Cyrus's recent research on content effort.

Google's Quality Raters are asked to judge qualities such as effort, originality and creativity. That can sound strange because Google cannot literally know how many hours someone spent creating a page. Cyrus's point was that it does not need a stopwatch. Google can learn what content judged to have genuine effort tends to look like.

That gets more interesting as producing competent generic content becomes cheaper. A page can contain the expected keywords and cover the expected topics without contributing anything particularly difficult to recreate.

Original research, first-hand evidence, strong opinions and real experience become more interesting in that environment. They give both users and machines information that cannot be reproduced simply by prompting another model for another summary.

A GEO win can still be a Google loss

Will then shared one of SearchPilot's published GEO experiments with Omio.

The change added a structured summary of important information near the top of a page. The hypothesis was straightforward: make useful information easier for an LLM to extract and summarise, and the pages may perform better in AI discovery.

They did.

The problem was Google.

The change performed positively for LLM traffic while Google organic performance moved in the wrong direction. SearchPilot's Omio GEO A/B testing story goes into the result in more detail.

A team looking only at AI visibility could have called the change a success. Measuring the wider search impact changed the rollout decision.

AI citations can be surprisingly easy to win

Cyrus has seen a similar pattern in his own experiments.

His AI Citation Ranking Factors Analysis brought together 54 of the strongest experiments, studies, explainers and patents he found. One of his biggest takeaways was that citations can be surprisingly susceptible to straightforward content changes.

Put relevant information high on the page. Make it easy to extract. Match the answer closely to what the system needs. Citation visibility can improve.

But the same intervention may not help Google. It can push more useful content lower down the page, introduce repetition or make the page better suited to extraction while making it worse in another part of the search journey.

That is why "more citations" cannot automatically mean "better search performance". SearchPilot's SEO vs GEO comparison tool is built around exactly that distinction.

A citation is not a recommendation

This was probably the most important distinction in Cyrus's research.

A citation tells you where an AI system found information that supports its answer.

That does not mean the system recommended the cited brand, site or product.

An ecommerce brand could provide a useful statistic and earn a citation while the model recommends a competitor. For the business, those are very different outcomes.

Cyrus worries that the industry is concentrating heavily on citations because they are visible and easy to count. Recommendations are harder to earn and harder to put on a dashboard. That does not make citations useless, but it does make them an incomplete definition of success.

There may still be a citation halo effect

Cyrus did mention one reason not to dismiss citations.

The Seer Interactive AI CTR Study found evidence that appearing in AI results alongside traditional organic visibility can affect click behaviour.

Cyrus was cautious about drawing too much from one study, but the possible mechanism is familiar. Seeing the same brand in more than one place can increase recognition and trust, similar to effects the search industry has discussed for paid and organic visibility.

So citations may contribute value even when the citation itself does not receive many clicks.

The mistake would be treating the visible citation as the entire outcome rather than one influence on a longer journey.

Fan-out and grounding queries do different jobs

Cyrus and Will also discussed a distinction that gets lost in a lot of AI search reporting: fan-out queries and grounding queries.

Fan-out queries are the searches an AI system may run to research a user's request. One prompt can trigger many background searches covering subtopics, comparisons and supporting facts.

Grounding queries are used to validate the answer. The system has a claim it wants to make and searches for evidence that supports it. Cyrus explained that many of the citations users can see are connected to this grounding stage.

That means visible citations only show part of the process. A site might influence the research phase without appearing in the final citations. Or it may win a citation because it validates a claim without having much influence over the recommendation itself.

The web still works better with links

Will pushed back slightly on the idea that AI mode and AI overviews will always have low click through rates.

His hope is that AI search interfaces get better at explaining why users might want to visit the underlying sources. The web works because pages link to other useful pages. An AI system can do the initial research without needing to collapse everything into one generated answer.

He gave a simple example involving recipes. The useful role for AI was to find recipes matching his requirements, then give him the original links. He still wanted the author's detail, notes and judgement rather than a generated average of several recipes.

Cyrus was less optimistic about how much traffic platforms will ultimately send, although he agreed that some AI search interfaces have already improved the way links appear.

Clicks matter more to Google than many SEOs thought

The discussion then returned to classic Google search and Cyrus's work on Google click signals.

For years, the SEO industry argued about whether user clicks influenced rankings. Information exposed through Google's antitrust cases, patents and leaks has given the industry a much clearer look inside.

Cyrus shared how behavioural data has played a substantial role for a long time. Google can observe what users click, whether they return quickly to search results, and whether a result appears to have satisfied the search.

That does not translate into a simplistic rule like "improve dwell time and rankings go up". The systems are more complicated than that. But it does mean the user's response to a result is much harder to dismiss than many SEOs once assumed.

Good clicks, bad clicks and long clicks

Cyrus broke some of those signals down into intuitive categories.

A bad click is the obvious failure case. Someone clicks a result, quickly decides it is not useful, and returns to the search results.

A good or long click is different. The person stays with the result. Cyrus's reading of the evidence suggests Google interprets that behaviour in context rather than applying one universal threshold.

The particularly interesting concept is the "last longest click": the user finds something that satisfies the search and does not need to return to keep looking.

Site owners cannot measure that perfectly from their own analytics, but it is a useful way to think about page quality. Did the page actually finish the search?

Why the industry got clicks wrong for so long

Will also raised the history here.

Rand Fishkin spent years arguing publicly that user behaviour mattered to Google's search systems and took a lot of criticism for it. Cyrus's view is that the industry's reluctance came partly from the lack of visibility into Google's systems and partly from taking Google's public wording too literally.

This is another example of why search teams should be careful with certainty. Something can be difficult to observe from the outside without being unimportant inside the system.

Rand's more recent Search Everywhere research broadens that point beyond Google. Search behaviour now happens across social networks, marketplaces, video platforms, AI tools and traditional engines, making the measurement problem even harder.

Answer quickly, then give people a reason to stay

There is an interesting tension between Cyrus's content ideas and click signals.

Cyrus likes the idea of answering the user's main question high on the page. That reassures the visitor immediately that they have found the right result.

But the page still needs enough value to keep the visit useful. Will described the problem as finding the right combination: make the primary value obvious, then provide enough depth, evidence or useful next steps that users do not immediately return to search.

That could mean original data, video, product reviews, comparison material or related questions. Different audiences will respond differently, which is why the exact implementation needs testing rather than another universal page template.

Knowing what worked matters more than explaining why

Will said SearchPilot has deliberately become less obsessed with identifying the exact mechanism behind every result.

It is interesting to know why something worked, but modern search systems are too complex to prove most explanations. A longer page might perform better because it targets additional queries, gives the user more information, improves behavioural signals, supports AI retrieval or interacts with several systems at once.

The commercially useful question comes first: did the change work?

Controlled testing can answer that with more confidence than a long story about what an algorithm probably did.

AI has broken the old attribution story

Cyrus thinks AI is pushing marketers back towards a less tidy model of attribution.

The old SEO story could be presented as a straight line: create content, rank, earn traffic, generate conversions.

Now a piece of content might influence Google, an AI answer, a YouTube search, social media, a later branded search or a direct visit without ever receiving the final click that gets credit.

That is uncomfortable for teams used to highly attributable channels. It also means that work can matter without producing a clean last-click trail.

Discovery is fragmenting faster than the attribution systems used to measure it.

Content marketing gets caught in the middle

Cyrus said attribution was one of the most common problems raised by marketers in a recent mastermind session.

That creates an awkward moment for content. Companies see organic traffic changing and start questioning traditional content investment. At the same time, AI systems still need information to train on, retrieve and use when generating answers.

Will described content as potentially getting caught in the worst of both worlds. The old attribution path is weaker, while the new path through AI discovery can be harder to see because content may influence training, fan-out searches or recommendations without sending a direct referral.

The work can still matter even when the dashboard cannot draw a neat line from article to conversion.

Prompt tracking is probably not the end game

The conversation closed on the gap between the questions leadership is asking and the tools search teams currently have.

Will has heard from senior SEO leaders that they have had more questions from executives about search in the last six months than in the previous ten years. AI Overviews, AI Mode, ChatGPT and Perplexity have pushed search visibility much higher up the organisation.

Prompt trackers have grown quickly because they give teams something they can show in response. But they only sample a small part of what is happening. They cannot capture every personalised prompt, fan-out query, hidden influence or downstream purchase.

So prompt tracking can be useful without being the final measurement model. SearchPilot's view is that teams should move beyond visibility alone and test the actual changes they are considering, measuring SEO and GEO together wherever traffic volumes make that possible.

Put Search in Control Mode with SearchPilot

Search is getting harder to observe at exactly the same time that leadership wants clearer answers.

That is a bad environment for checklists.

SearchPilot helps enterprise teams make SEO and GEO testable. It runs controlled experiments across high-impact site sections and measures Google organic and LLM performance together, helping teams identify where a change genuinely works and where an apparent AI win hides a larger downside.

The Omio experiments show why this matters. One GEO hypothesis increased LLM traffic. Another performed positively for AI discovery but would likely have hurt Google organic sessions enough that it was not rolled out.

You don't need more SEO or GEO tactics. You need to know which ones actually work.