What model collapse is, what the experiments behind it tested, and a separate concern for AI review.
Model collapse is the loss that follows when generative models are trained on output from earlier models: the rare parts of the original data fade first, and probability gathers on common forms. The term comes from a 2024 Nature paper, and it is often read as a forecast that AI will poison itself as the web fills with machine-written text. The evidence supports something narrower. It also points to a separate concern for anyone who relies on AI to check work: models that share their mistakes.
What the experiment showed
Shumailov and colleagues trained models on the output of the previous generation, again and again. They report “irreversible defects in the resulting models, where tails of the original content distribution disappear”, in three kinds of model, language models among them. The unusual goes first; the common takes its place.
Philologists know the pattern from manuscripts. Copyists tend to replace a difficult reading with an easier, more familiar one, and over many copies the unusual wording disappears. Our research paper calls model collapse the purest measured case of that mechanism, under the conditions of the experiment.
The doom version rests on one set-up
In those experiments each generation was trained on the previous generation’s output in place of the original data. Gerstgrasser and colleagues compared that with a set-up in which synthetic data accumulates alongside the real data. Replacing the real data tended towards collapse; accumulating avoided it, across the model sizes and architectures they tried. A follow-up found the same for accumulation, and slow, gradual decline where each generation trains on a fixed-size subset of the growing pool.
The question is not settled. Dohmatob and colleagues argue, in a theoretical regression setting with supporting experiments, that “even the smallest fraction of synthetic data (e.g., as little as 1% of the total training dataset) can still lead to model collapse”. And a 2025 position paper counts eight distinct, sometimes conflicting definitions of model collapse in the literature, and concludes that some predicted collapse claims rest on conditions that poorly match the real world, and that several prominent collapse scenarios are readily avoidable.
So “AI is eating itself” is a disputed forecast, not a finding. Laboratories also filter their data and generate synthetic data deliberately, which these experiments do not reproduce exactly.
What is better supported: less variety
When language models are fine-tuned repeatedly on their predecessors’ text, Guo and colleagues report “a consistent decrease in the diversity of the model outputs through successive iterations”, lexical, syntactic and semantic, strongest on creative tasks. Briesch and colleagues, using tasks whose answers can be checked exactly, find that the loop “produces correct outputs, however, the output declines in its diversity depending on the proportion of the used generated data”; fresh data can slow the decline but not stop it.
That is a different loss from the one in the headlines. A model can keep giving correct answers while giving them in fewer ways. And recursion may not be the only cause: a 2026 study argues that the sameness of model output is likely learned in pretraining and only revealed or amplified by alignment, and finds that prompting alone can bring it out in models that were never aligned.
And models that fail alike
Variety matters most where models are wrong. A study of more than 350 models found substantial correlation in their errors: on one benchmark, when two models were both wrong, they gave the same answer 60% of the time, and larger, more accurate models had highly correlated errors even across different architectures and providers (correlated errors). A 2026 simulation of AI trading agents found their behaviour grew more correlated as capability rose; adding agents reduced risk when their shared reasoning was right, and the same correlation became a liability when they shared false information. Its authors leave open whether the same happens in other domains.
Our research paper argues that training on the growing body of machine-written text is a second pressure in the same direction: models inherit material from earlier models through transmission, and shared inheritance may mean shared blind spots. That link is argued from how the channels work. It has not been measured as a trend.
What this means for AI review
If models are becoming more alike, two AI reviewers agreeing on your code is weaker evidence than it looks: the second reader may be a second draw from the same pool of habits. That is why we count model families rather than vendors (see A new vendor is not necessarily a new seat), and why we add checks that cannot share the models’ blind spots: a test that runs, a compilation, a proof, an oracle in our terms.
Research on training points the same way. Feng and colleagues report that verifiers, even imperfect ones, can select generated data that prevents collapse. A 2025 analysis adds a limit: a model retrained under a verifier’s guidance converges towards what the verifier knows, and its theory predicts that, unless the verifier is perfectly reliable, early gains will level off and may reverse, which its experiments confirm. A check helps as much as it is independent and right.
A new preprint tests self-review directly in a code-training loop. Song, Cai and Zhao fine-tuned four small code models on their own code for five rounds, filtering it with no check, with Python’s compile check and simple text rules, or with the model scoring its own code. Every version ended below where it started. The paper suggests that stable training requires checks that do not depend on the model; our CounterResearch review found that its own results do not show the model’s self-scoring doing worse than the cheap fixed checks, and that each result comes from a single run. What it does show is that neither a model’s own judgement nor cheap automatic checks stopped the decline.
What our research proposes
Our research paper does not propose a test for model collapse. Two of its proposals are narrower. One is an archive of sealed earlier open-weight models: “Preserved vintages would make changes in the corpus testable against fixed references.” The other is comparing models with each other, which “also asks how differently they fail”; that bears on models that fail alike, not on correct answers given in fewer ways. These are proposals; the paper says it “does not itself establish that the instruments will work”.
Questions this page answers
What is model collapse? Model collapse is the loss that follows when generative models are trained on output from earlier models: the rare parts of the original data fade first, and probability gathers on common forms. It was shown for models trained generation after generation on their predecessors’ output.
Is AI getting worse because of synthetic data? That is not established. The strongest collapse results come from experiments in which each generation replaces the real data with synthetic data; one study and a follow-up find collapse avoided when synthetic data accumulates alongside the real data, though training each generation on a fixed-size subset of that pool still declines slowly; another study, with theory and experiments, finds that even about 1% synthetic data can still cause collapse in its setting. Less variety in model output is better supported than a general decline in quality.
Does model collapse affect AI code review? Not as a sudden failure. The concern for review is that models can share mistakes, so that two AI reviewers agreeing on your code can be weaker evidence than it looks. A check that does not use the models, such as a test that runs, helps as much as it is independent and right. A second model is not automatically a second opinion, though the studies do not show that more models never help.
What we have not shown
We have not shown that the models in use today are collapsing, or that their errors are converging because of machine-written training data; the second is our paper’s argument, not a measurement. The share of machine-written text on the web is itself disputed. Whether defective code habits are passed on through public repositories into later models is an inference we have marked for testing. For the papers cited here we read the abstracts, and Song, Cai and Zhao in full.
The questions worth asking of any multi-model review are both whether its models are good and whether they are still different.
Sources (read 18 Oct 2026; abstracts unless stated):
- Shumailov, Shumaylov, Zhao, Papernot, Anderson & Gal, “AI models collapse when trained on recursively generated data”, Nature 631:755, 2024; earlier as arXiv:2305.17493 (quotation from the arXiv abstract).
- Gerstgrasser et al., “Is Model Collapse Inevitable?”, arXiv:2404.01413. Kazdan et al., “Collapse or Thrive?”, arXiv:2410.16713.
- Dohmatob, Feng, Subramonian & Kempe, “Strong Model Collapse”, arXiv:2410.04840.
- Schaeffer, Kazdan, Arulandu & Koyejo, “Position: Model Collapse Does Not Mean What You Think”, arXiv:2503.03150.
- Guo, Shang, Vazirgiannis & Clavel, “The Curious Decline of Linguistic Diversity”, arXiv:2311.09807 (NAACL 2024 Findings). Briesch, Sobania & Rothlauf, “Large Language Models Suffer From Their Own Output”, arXiv:2311.16822.
- Fortier, Chen & West, “Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models”, arXiv:2608.11426.
- Kim, Garg, Peng & Garg, “Correlated Errors in Large Language Models”, ICML 2025, arXiv:2506.07962. Ross et al., “Why Better Models Can Create Riskier Systems”, arXiv:2609.04373.
- Feng, Dohmatob, Yang, Charton & Kempe, “Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification”, arXiv:2406.07515. Yi, Liu, Cheng & Xu, “Escaping Model Collapse via Synthetic Data Verification”, arXiv:2510.16657.
- Song, Cai & Zhao, “When AI Reviews Its Own Code”, arXiv:2606.28438v2 (read in full); our review on CounterResearch.
- Our research paper, “Agentic Stemmatics”, §§6 and 7.3: research.counterproof.io/agentic-stemmatics.html.