Google didn’t want to give publishers the choice to keep their content out of AI Search results because it’s “evolving into a space for monetisation.” That’s according to a newly disclosed internal document spotted by Bloomberg, which reveals that Google had discussed offering publishers more granular control over how website data would be used in AI Search features instead of the illusion of choice they eventually received.
The document, written by Google Search executive Chetna Bindra, was released during the US antitrust trial into Google’s online search monopoly. The access to its search engine data gives Google a huge advantage in AI development over rivals like Perplexity and OpenAI. But Google’s AI Overviews and AI Mode can be detrimental to the websites they source from by reducing clickthroughs, incentivizing publishers to keep their content out of AI summaries and related features if given the choice.
One of the suggestions in the documents that Google considered a “hard red line” would enable publishers to prevent Google’s AI models from referencing their data in real time, but not opt out of being used to train features like AI Overviews generally. Another option, labelled as “likely unstable,” suggested that no additional controls should be added, and that publishers can opt out of being indexed on Search entirely, “if not satisfied.”
A court hearing on May 2nd revealed that publishers are facing that ultimatum. While Google introduced a way for publishers to opt out of AI training in 2023, Google DeepMind vice president Eli Collins said it doesn’t apply to search-specific AI products like AI Overviews. The only way for publishers to avoid AI Overviews sucking up their content is to opt out of being crawled by Googlebot — which stops their website being indexed for Search altogether.
When AI Overviews rolled out last year, Google decided to “silently update” the information about publisher controls with “no public announcement,” according to the document. Guidance on how to word the update also suggests that Google intentionally made it harder for publishers to know what they were actually opting out of to avoid getting ”into the details of distinction” between training for Gemini, AI Overviews, and other AI models.
“Do what we say, say what we do, but carefully,” Bindra said in the document.
Google says that this document was an early list of options it was considering as AI search was evolving, and doesn’t reflect the decisions it ultimately made. “Publishers have always controlled how their content is made available to Google as AI models have been built into Search for many years, helping surface relevant sites and driving traffic to them,“ Google spokesperson Peter Schottenfels said in a statement to The Verge. ”New search features like AI Overviews have led to more searches, which creates new opportunities for sites to be discovered.
The wording that Google currently uses is more upfront, saying that publishers who flag their content not to be used for AI Overviews and AI Mode will also keep it out of “all forms of search results.”
