In plain words
AI models learn from huge amounts of human-made text and images. But more and more of what is online is now made by AI. Future models, trained on the web, will learn partly from earlier models' output.
Researchers tested what happens when you repeat that loop. Each model learns slightly imperfect patterns and over-produces the most common things. The next model learns from that, and loses a little more. Rare words, unusual facts and minority cases fade first. After many rounds, the output becomes repetitive and strange.
This is model collapse. It is not inevitable. Studies find it is avoided when real human data is kept in the mix instead of being replaced. The synthetic data page explains when AI-made data helps.
Everyday example
The forward of a forward
A photo is shared in a WhatsApp group. Someone screenshots it and forwards it. Someone else screenshots that and forwards it again. Each copy loses a little: first the fine details in the corners, then the small text, then the colours. Ten forwards later you get a blurry blob that only roughly matches the original. Model collapse is the same slide, happening to an AI's picture of the world.
Where the example breaks: A forwarded photo only loses detail. A collapsing model also adds its own mistakes and shifts towards whatever is most common, and the loss can be avoided by mixing the original back in at every step.
How it actually works
The best known study is by Ilia Shumailov and colleagues at Oxford, Cambridge and other universities, published in Nature in July 2024.
The loop
Train a model on real data. Use it to generate new data. Train the next model on that, and repeat. The study defines model collapse as a process in which generated data "end up polluting the training set of the next generation", so models "mis-perceive reality".
Early collapse: the tails go first
Any model learns from a limited sample and tends to favour common patterns. Rare events, the tails of the data, are under-sampled, then missed entirely. The paper notes these rare cases often matter most to minority groups.
Late collapse: drift
Errors pile up over generations until the output barely resembles the original data and varies much less. In one test, a small language model was given a passage about church architecture. By the ninth generation it was listing jackrabbits with different coloured tails.
Not just text
The same effect showed up in image models and simple statistical models. A 2023 study named the image version Model Autophagy Disorder, or MAD, after mad cow disease, and found models need enough fresh real data each round.
How to avoid it
In the Nature tests, keeping 10% of the original data at each round led to only minor decline. A 2024 study found that accumulating new synthetic data alongside the original real data, instead of replacing it, avoided collapse. Labs also filter generated data and track where data comes from, which is hard because AI content online is rarely labelled.
- Named
- May 2023 preprint "The Curse of Recursion", published in Nature on 24 July 2024 as "AI models collapse when trained on recursively generated data"
- Key finding
- Indiscriminate training on model-made content "causes irreversible defects" in which the tails of the original data disappear
- Nicknames
- AI inbreeding, Habsburg AI
- The fix in short
- Keep real human data, add rather than replace, and check generated data
The researchers argue human-made data and records of genuine human interactions will become more valuable as AI content spreads online.
Is model collapse a real threat to future AI?
One view
Yes, the web is filling up
As AI content floods the internet, future web-scraped data will contain more of it, and it is hard to label at scale. The Nature authors warn this must be taken seriously, and suggest early models trained on mostly human data may hold a lasting advantage.
The other view
It is avoidable in practice
Collapse shows up mainly when real data is thrown away and replaced. A 2024 study found that keeping real data and adding synthetic data on top avoids it, and builders can filter data and keep large human-made collections.
Most likely: Both findings hold. Collapse is real when training on AI output is careless and repeated, and avoidable with real data kept in the mix. How much AI content in tomorrow's web data will matter is still being studied.
Questions people ask
What is model collapse in simple words?
When AI models are trained on content made by earlier AI models, over and over, they slowly forget rare and unusual things and drift away from reality. Each round copies the last one's blind spots and adds new mistakes.
Is model collapse happening to ChatGPT and other chatbots?
There is no public evidence that today's major chatbots have collapsed. The studies used controlled experiments with smaller models, repeated many times. Keeping and filtering real human data, which builders can do, prevents collapse in the research, but how it plays out at full scale is still being studied.
Why is it called Habsburg AI?
It is an informal nickname, like "AI inbreeding": a model learning from its own family line instead of from fresh outside data. The comparison is loose and not a scientific term, but it captures the idea of a closed loop slowly losing variety.
Is training on synthetic data always bad?
No. Carefully made and checked synthetic data, mixed with real data, is widely used and helps many models. The problem is careless, repeated training where AI output replaces real data instead of adding to it.
What is the difference between model collapse and data poisoning?
Data poisoning is a deliberate attack: someone plants bad examples on purpose. Model collapse is accidental: it comes from models feeding on their own output over generations, with nobody intending harm.
Further reading
- AI models collapse when trained on recursively generated dataNature, 2024
- The Curse of Recursion: Training on Generated Data Makes Models ForgetarXiv, 2023
- Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic DataarXiv, 2024
- Self-Consuming Generative Models Go MADarXiv, 2023