AI models are cannibalizing their own content. This self-referential spiral sees Large Language Models (LLMs) like ChatGPT and Google Gemini consuming the very material they were trained on.
The issue started when human content became scarce and AI-generated output took over. Veteran tech journalist Steven Vaughn-Nichols warns of “model collapse.” This happens when AIs trained on AI outputs produce irrelevant or outright erroneous content, a perfect example of “Garbage In/Garbage Out.”
"AI output loses coherence, accuracy drops off a cliff, and the models start spitting out content that ranges from offensively wrong to just plain offensive." — Steven Vaughn-Nichols
To combat this, companies are adopting a technique called retrieval-augmented generation (RAG). This allows AI to "Google" real-time information instead of solely relying on outdated data. The goal? Keep AI models sharp with fresh, human-written insights.
However, a recent Bloomberg study shows a troubling trend. RAG-enhanced models generated more unsafe or unethical responses compared to traditional models.
"The RAG models were far more likely to generate unsafe or unethical responses, ranging from privacy violations to outright misinformation."
This is especially concerning given their use in customer service and health advice. Vaughn-Nichols suggests this could lead to a “slow-motion car crash” for AI. Without fresh human-produced content, stagnation is inevitable.
The existence of LLMs hinges on human creativity to generate original ideas. As AI continues to consume itself, it raises a critical question: How long until the well runs dry?
Unless companies incentivize ongoing human content creation, the current AI boom may be on borrowed time.
