Search is changing, but this isn't the first time it's changed, and it isn't the first time the way we search has changed either. In this article, I reflect on previous changes, and what it means that we are seeing an explosion in search volume at the same time as queries change from keywords to paragraphs (or more) and click through rates decline.
Prefer to watch?
Starting at the beginning
Back in 2009, I stood on a stage and, with the confidence of a man in his twenties with a microphone, told the story of a restaurant owner who'd asked me recently about ranking for the single word query [breakfast].
"Who searches like that?" I said. "What kind of context-free lunatic just types [breakfast] into a search bar and expects it to work?".
By 2016, it was my turn to be the fool.
I was in Boston for a conference, jet-lagged and in a city I didn't know well. I pulled out my phone, opened the search bar, and typed one single, desperate word:
[breakfast]
And. It. Worked.
It was magic. My phone knew where I was. It inferred I had an urgent need for coffee and eggs (rather than, say, a recipe or god-forbid a wikipedia definition of breakfast). It presented me with well-reviewed nearby places that were open (it knew what time it was even if I didn't), ready to set me up for the day. The very thing I had declared absurd had become my reality. The computer could finally help.
This tiny moment of personal humiliation wasn't just about me and my confusion. It was a tremor heralding a seismic shift that was coincidentally the subject of my talk that day in Boston. I was giving a presentation entitled the emerging future of search. The better title, though, was buried on slide 121:
I should have called it the next trillion searches. The presentation was the result of detailed research and lots of thinking by me and Tom Anthony - my colleague and now SearchPilot's CTO. We pointed out that the history of search can be measured in these trillion-query-a-year chunks:
- The first trillion searches: happened at a desk. You sat at a computer and you were mainly hoping to see web pages that contained the words you had searched for.
- The second trillion searches: the mobile explosion. You pulled a supercomputer from your pocket and hoped to see web pages that matched what you were implicitly searching for. My pathetic plea for breakfast was a drop in the trillion-search-a-year ocean, and it was made possible because my phone had my implicit context - my location, my search history, etc.
- Our prediction about the third trillion searches: in that 2016 presentation, we predicted the next trillion searches a year would come from expanding the implicit context. We talked about answering questions people weren't even asking yet, because they just assumed computers were too dumb to handle them. We thought this would happen through deepening implicit context.
Looking back now, we got two big things wrong, but one thing very right. We were right that there was a huge opportunity coming. Our prediction that it was the next trillion searches, however, feels... quaint. Adorable, even. We totally under-egged it. Google alone has now blown past the five trillion searches a year mark (and the wider AI search ecosystem is also on fire). We saw a trillion-searches-a-year opportunity. It's turning out to look more like 10 trillion.
We also got the context thing the wrong way round.
From what the computer knows to what you can tell it
It turns out the next great leap wasn't about the computer inferring more about us. It was about us finally being able to tell the computer everything. We have rocketed past the era of implicit context and slammed headfirst into the age of extreme explicit context.
This chart explains it all:
In 2009, [breakfast] was a useless query. By 2016, it was magic. But today, we're somewhere else entirely. Today's query isn't a query; it's a short novel. It's a hyper-specific demand directed at a deeply literal but shockingly competent personal assistant.
[breakfast restaurant suitable for a business meeting within a 10-minute walk of the eurostar platforms at st pancras with good vegan options but also offering a traditional full english breakfast for different members of the party. if there’s an opportunity to choose a restaurant with a view of central london, we’d take it, but prioritise food reviews over the view]
In 2016, this query was gibberish. You would have found no (good) results. No single web page on the entire internet was likely to contain enough information to match that exact string of requirements. But today, an LLM doesn't need to find the perfect page.
It just creates the perfect answer. (And if you're lucky that answer will correlate to something that actually exists in the real world and isn't a computer's fever dream hallucination).
This changes everything. This isn't just a bigger version of what came before. This is a new kind of search.
This isn't a prediction. It's happening right now.
You don't have to take my word for it. Just listen to Google's CEO, Sundar Pichai. On a recent earnings call, he dropped this little nugget:
AI Overviews are driving over 10% growth in the types of queries that typically display them
And on the All-In podcast, he was even more direct:
people are typing in queries like literally long paragraphs
What it means for marketers
This explosion of new, complex, conversational queries is the biggest disruption to come along since mobile. (And for ecommerce and retail sites, who face far fewer of the downsides, it should be a boon).
But this new world comes with two massive headaches for marketers:
- Your clicks are different now. The click-through rate on these AI-assisted research queries is way, way lower. Comparing this year's clicks to last year's is an apples-to-sentient-oranges comparison. The clicks you get next year are not going to be the same as the clicks you got last year. Stop pretending they are.
- It's a black box. The whole process is infinitely more complex and opaque than old-school search ever was. The AI might be performing multiple background searches ("query fan out", if you want to sound fancy) before it even thinks about showing a result. Trying to understand the inner workings from the outside is basically impossible.
So what the heck do we do?
When you're facing a black box, you have one reliable strategy: scientific testing. You can't know what's happening inside, but you can meticulously measure the inputs and outputs.
And look, full disclosure, this is the part where I tell you about my company.
At SearchPilot, we've been building the tools for this new reality since we first started talking about the next trillion searches. Our entire approach is designed for a world where neither we nor you know the rules.
It's built to:
- Work against a black box where the inner workings are a secret.
- Be robust to all the weird behind-the-scenes research the AIs are doing before they recommend you (or your competitor)
- Integrate a whole range of data - from clicks to impressions to overall visibility - to figure out what's actually working.
We've been building for this moment. The next ten trillion searches are here. They are long, they are specific, and they are messy. It's time to start experimenting.
