Facepalm: As Google's AI Overviews (AIOs) kill traffic to sites and AI-generated content floods the web, it has long been predicted that these features would eventually run out of human-made data and begin citing information created entirely by AI. Sure enough, it's starting to happen: a study found that around 10% of AIO sources are AI-generated.
One of the many concerns about so much AI slop appearing online is that other AI services, such as AIOs, would start citing it. Given the technologies' penchant for making thing ups and the ramifications of AIs learning from AIs, it's a worrying prospect.
A recent study by Originality.ai, which makes software that detects if something was created by AI, is giving dead Internet theory believers more support for their argument.
The company randomly sampled 29,000 Your Money or Your Life (YMYL) Google queries – topics that could significantly impact a user's health, safety, financial stability, or happiness. It then analyzed the AIOs that appeared at the top of Google's search page, the links they cited, and the first 100 organic results for each query.
Using its AI Detection Lite 1.0.1 model, Originality.ai concluded that 10.4% of the AIOs were likely generated by an AI.
Concerns about AI feedback loops – AI models learning from other AI models – go back years now. In 2023, a team of researchers from the UK and Canada published a paper (featuring the AI-generated title The Curse of Recursion) on this phenomenon and how it causes "irreversible defects." It coined the term "Model Collapse."
The new study notes that its findings show a long-term risk of model collapse. It adds that AI Overviews themselves are not part of training data, but by surfacing AI-generated sources, they boost those sources' visibility and credibility. This increases the likelihood that such material will be used in future training sets.
Unsurprisingly, Google objects to the study's findings, arguing that the accuracy of Originality.ai's AI detector is flawed.
"This is a flawed study relying on partial data and unreliable technology," a spokesperson told The Reg. "AI detectors have not proven their effectiveness at detecting AI generated content – in fact, many have demonstrated they are error-prone. As in Search more broadly, the links that are included in AI Overviews are dynamic and change based on the information that is most relevant, helpful, and timely for a given search."
While AI-detection software admittedly isn't perfect, tests have shown that Originality.ai's tool has received high marks for accuracy.
The rest of Originality.ai's study showed that 74.4% of the AI citations were human-written. 15.2% were "unclassifiable," mostly because they had too little text, broken links, or were video pages/PDFs.
It was also found that of the citations that did work in the AIOs, 52% were not among the top 100 pages Google showed in its organic search results for the same term. And 12.8% of these -- higher than the overall number -- were flagged as AI-generated.