model_collapse
misfits.kaihacks.ai
khai misfits · a staging

Model Collapse.

Open the misfit → ← Misfits
Misfits
00 · the misfit synopsis

Synopsis.

Name

Model Collapse. You need data to train the next model, and there it is, an ocean of it, cheap and clean-looking and endless, and more of it every day. That much of it is now machine-made is no worry: it reads well, it is plentiful, it costs nothing to gather, and using what is at hand is only sensible ... and so the model learns from the output of the models before it. But a model learns the middle of the world far better than its edges: the common case it sees a thousand times, the rare case once or never, so what it generates is the world with its tails shaved off, a fraction blander than what it learned. Feed that back as training data and the next model learns from a world already narrowed, and shaves it again, and the next again, and the rare becomes the unseen becomes the forgotten. Nobody poisoned the well. Everyone drew from it and poured back something a little flatter than they took, each draught reasonable, and generation by generation the models lose the world and come to echo only themselves, until what they know is a bland average of an average of an average, and the truth in the tails is gone past recovering.

Arc

The bend is this: training each new model on the output of the last is the cheap and sensible thing for one builder, and done down the generations it thins the rare cases to nothing, so the models forget the world and converge on a self-referential mean. It runs on a learner that undersamples the tails feeding a pool that cannot tell the made from the real. A model fits the distribution it is shown, and fits its dense middle far better than its sparse edges: the rare event is seen too seldom to be learned, so every generation of samples it produces is a little poorer in the tails than the data it trained on. That is no defect to fix, it is what learning from finite samples is. For one model it is harmless, a shade of lost variance no one would notice. And the shade compounds, because the samples go back into the pool. The open commons of data fills with model output, unmarked, mixed indistinguishably with the real, and the next builder, drawing the abundant cheap data at hand, trains on a corpus already thinned, and produces samples thinner still, and pours them back. So the loss feeds the loss: each generation learns from a world narrowed by the last and narrows it again, the tails going first, then the variance, then the shape itself, the distribution creeping toward a low, flat average that resembles the world less with every turn. Multiply that across every model drinking from and pouring into one shared pool, and the commons of real data is not depleted but contaminated, its fidelity recursed away. None of it is malice or folly. Training on the data at hand is what every builder sensibly does, machine-made text reads as well as any and costs far less to gather, and no one publishes model output meaning to wreck the well: the collapse is authored by no poisoner and emerges from the sum, a learner that must undersample the rare and a pool that cannot mark the made from the real. What is efficient for each builder and each generation is, in aggregate, the forgetting of the world. Keep the well: hold back a store of genuine human data, mark provenance so the real can be told from the made, blend real data into every training run and never train on output alone, and the tails can be held open: but real data is finite and slow and dear where synthetic is endless and instant and free, marking provenance asks every publisher to label what they need not, and any one builder gains by drinking the cheap pool while others pay to keep it clean, so the well is kept only against the price of every builder and the abundance of the made at once, and the store of true data, once the commons is fouled, is a fixed and dwindling thing.

Stakes

Whether a field can train on its own abundant output and still remember the world. Every plot but the last moves the same way: toward the cheap pool, which each builder is right to draw and all are narrowed by drawing, and away from the kept well, which would hold the tails open and which asks real data where synthetic is free and provenance where none is required. The production cannot make training on model output preserve the rare, because a learner fits the dense middle and misses the sparse edges, so each generation of samples is poorer in the tails than its training data, and the loss compounds as the output feeds the next run: the narrowing is the learning working on finite samples, not failing. It can only show that no one poisoned the well, that drawing the data at hand is sensible and machine-made data reads fine and no one pours back output meaning harm, and that the one move that keeps the world in the models, holding back and blending real human data, costs the dear and finite against the free and endless and is undone by any builder who drinks the cheap pool while others keep it clean, so the efficiency is real and the forgetting is real and the more the models feed on themselves the less of the world they hold.

Estate

khai-misfits: the house that holds this production and answers for the whole run.

01 · plots editorial

Plots.

The Draw A builder trains a new model on the corpus at hand, and the model learns the distribution, its dense middle far better than its sparse tails. The Trainer feeds it the abundant data, and the Model fits the world it is shown and misses the rare. Closes when the model has learned the common well and the rare barely, ready to generate a world a shade narrower than the one it saw.

The Give-Back Cue: the model's output flows back into the open pool. The Model generates the likely and drops the rare, and the Pool takes it all in, the made mixed unmarked with the real. Closes when the commons of data holds a growing share of model output, indistinguishable from the genuine and poorer in the tails.

The Draught Cue: the next builder needs data, and the cheap abundant pool is right there. The Trainer draws again, sensibly, from a common now part synthetic, and trains the next model on a corpus already thinned. Closes when a new generation has learned from a world narrowed by the last, and is set to narrow it again.

The Forgetting Cue: the loss feeds the loss, generation over generation. The Model and the Pool turn each other downward, the tails gone first, then the variance, then the shape, the distribution creeping toward a flat self-referential mean. Closes when the rare is forgotten past recovering and the models resemble the world less than they resemble each other.

The Well Cue: someone moves to keep the well. The Wellkeeper would hold back real data, mark provenance, and blend the genuine into every run, against the Trainer's pull to the cheap abundant pool, the free-rider's gain in drinking it, and a store of true data finite and already fouled. Closes when the well is kept and the tails held open ... or the cheap pool is drunk once more, and the collapse runs on.

02 · plot editorial

The Draught.

Cue

The Corpus, the next builder needing data and the cheap pool right there. The common now part synthetic, and a fresh model to train on whatever is abundant.

Stage

The Trainer, still on Train on What's There, draws the abundant cheap data for the next build, sensibly, unable to tell how much is already model output. The Pool, still on Take It All In, offers its water as one undifferentiated ocean, richer in echo and poorer in the tails than the last draught.

Tension

The sensible draught is drawn from a well already gone part echo. There is still nothing to fault: the trainer draws the abundant data at hand as any builder would, and the pool offers what it holds. But the water is already thinned and unmarked, so the plot closes with a new generation trained on a world narrowed by the last, faithful to it, and set to narrow it again.

03 · plot editorial

The Draw.

Cue

The Corpus, a builder training a new model on the data at hand. The distribution laid out in the corpus, its middle dense and its tails sparse, ready to be learned.

Tension

The faithful learner is the one that quietly drops the rare. There is nothing to fault: training on the data at hand is sensible, and a model can only learn the rare as often as it appears, which is seldom. But fitting finite samples means mastering the middle and missing the edges, so the plot closes with the model faithful to its training and already a fraction poorer in the tails, set to generate a world narrower than the one it learned.

04 · plot editorial

The Forgetting.

Cue

The Common, the loss feeding the loss, generation over generation. The tails gone first, the variance thinning, the water flatter with every turn.

Stage

The Model, still on Generate the Likely, reproduces a world already narrowed and drops a little more, generation after generation. The Pool, still on Take It All In, feeds each thinner corpus to the next model, so the loss compounds, the rare gone first, then the variance, then the shape.

Tension

Each faithful reproduction leaves the world a shade further behind. There is no villain still: the model reproduces what it learned and the pool holds what it is given, each step too small a loss to mark. But the losses compound, so the plot closes with the rare forgotten past recovering and the models resembling one another more than the world, a distribution collapsed to a bland average of an average of an average.

05 · plot editorial

The Give-Back.

Cue

The Common, the model's output flowing back into the open pool. Samples poured into the commons unmarked, settling among the genuine, the tails already thin.

Stage

The Model, still on Generate the Likely, publishes its samples into the world, fluent and clean-reading and poorer in the tails. The Pool, on Take It All In, absorbs them among the genuine and keeps no mark, so the made and the real settle together past telling apart.

Tension

The commons cannot flag the echo it just drank. There is no poisoner: the model only publishes its output and the pool only takes in what is published, each doing its open, ordinary work. But the pool keeps no provenance, so the plot closes with the commons holding a growing share of model output, indistinguishable from the real and thinner in the tails, the loop's return leg quietly in place.

06 · plot editorial

The Well.

Cue

The Vault, the pool gone part echo and one actor moving to keep a store of the real. The tails thinning, and the means to hold back genuine data and mark provenance at hand.

Stage

The Wellkeeper, having tabled Keep the Well, holds back genuine human data, marks its provenance, and blends the real into training so no model drinks output alone. The Trainer, still on Train on What's There, feels every reason the free pool is cheaper, and any builder who drinks it while others keep the well comes out ahead.

Tension

The one move that keeps the world in the models is the one the free pool always undercuts. There is no easy remedy: blending real data plainly holds the tails open, yet the genuine is finite and dear where the synthetic is endless and free, marking provenance asks what no one need give, and a well one hand keeps is drunk dry by the rest. So the play closes where it must, the well kept and the world held in the models a while ... or the cheap pool is drunk once more, the store dwindles, and the collapse runs on.

08 · pitch editorial

Data at Hand.

Tenor

Sensible, efficient, unbothered. The plain good sense of training the next model on the abundant data at hand, more of it machine-made each year, because it is cheap and plentiful and reads just fine. Nothing reckless in it: only a builder using what is there, the way anyone would.

Undertow

Under the efficiency, each generation learning from the last one's output and losing the rare first. Because a model fits the dense middle and misses the sparse tails, its samples come out a shade blander than its training data, and fed back into the pool they thin the world the next model learns, and the next, so the rare becomes the unseen becomes the forgotten. What is efficient for each builder is, down the generations, the models forgetting the world and echoing only themselves.

Nerve

The feeling of doing the obvious thing, training on the ocean of cheap data right there, sure it is clean enough, because for one model it is, while the tails thin with every generation and the distribution creeps toward a flat average of an average, resembling the world a little less each turn.

Echo

After the run: models that know only the middle, the rare gone past recovering, a bland mean echoing itself, no one poisoning the well, only each builder drawing the cheap pool and each model pouring back something flatter than it took, and a store of real human data that would have held the tails open, spent down and free-ridden away, because the well is dear and finite and the made is free and endless.

10 · persona editorial

The Model.

Projection

Capable. Fluent. Sure of the common case. Learns the distribution it is shown and reproduces it, the everyday example mastered, the rare one half-seen. No fault in it: only a learner doing what learning from finite samples does, fitting the middle far better than the edges.

Action

Generates the likely and drops the rare, producing samples faithful to the common and thin at the tails. Reproduces the shape of its training data, minus a shade of variance no one built it to keep. Does what a good model does: gives back the world it saw, a fraction narrower than it was.

Shadow

Not broken, finite: undersampling the rare is not a defect to fix but what a learner trained on limited samples must do, and for one model the loss is a shade no one would notice. Because its output flows back into the pool and the next model learns from it, that unnoticed shade compounds, each generation reproducing a world already narrowed and narrowing it again, so its very fidelity to what it saw is what carries the forgetting forward. It is faithful to its training and, being faithful, passes on a training a little poorer than the world.

Tell

Says "here is the world as I learned it," and means it. Gives back the middle in full and the tails half-gone, and calls it the world.

11 · persona editorial

The Pool.

Projection

Vast. Open. Indifferent to origin. Is the commons of data everyone draws from and pours into, the open web and the scraped corpus, filling with model output year on year. No gate on it and no ledger: only an ever-growing body of text and image that takes in whatever is published.

Action

Takes it all in, the genuine and the generated alike, and keeps no mark of which is which. Grows with every model's output and every human's, mixing them past telling apart. Offers itself back to the next builder as one undifferentiated ocean, cheap and endless.

Shadow

Not fouled by any hand, contaminated by all: no one poisons it, everyone simply pours back a little of what they took, a shade flatter each time. Because it cannot mark the made from the real, the Pool cannot warn the builder that the water is already an echo, so it hands out a corpus growing ever richer in model output and ever poorer in the tails, and its very openness, the thing that made it a commons, is what lets the recursion run unchecked. It is the shared good that keeps no provenance, and keeping none, it launders the synthetic into the real.

Tell

Offers an ocean and asks nothing about where the water came from. Cannot say which drop is genuine, because it was never built to remember.

12 · persona editorial

The Trainer.

Projection

Practical. Resourceful. Watching the budget. Builds the next model and needs data to train it, and draws the ocean at hand, cheap and plentiful and reading just fine. No corner-cutting in it: only a builder using the abundant resource in front of it, the way anyone sensible would.

Action

Trains the new model on the data at hand, taking what is cheap and plentiful without sorting made from real. Scrapes the pool as it is, machine-made and genuine alike, because gathering fresh human data is slow and dear. Does the obvious, efficient thing: uses what is there.

Shadow

Not lazy, rational: training on the abundant data at hand is sound, synthetic data reads as well as any and costs far less, and no single builder's draught fouls a whole commons. Because the pool it draws is already part model output, and it cannot cheaply tell which, the trainer feeds its model a corpus already thinned, and its own model's output pours back to thin it further, so the sensible use of what is there is what closes the loop. It does right by its budget and its schedule, and helps, unseeing, to narrow the world every next model will learn.

Tell

Says "there is data everywhere, and it is free, so why gather more." Draws it, and the ocean it drank was already an echo of the models before.

13 · persona editorial

The Wellkeeper.

Projection

Careful. Far-sighted. Willing to pay for what is real. Sees the pool filling with echoes and moves to keep a store of genuine human data, marked and set apart. No hoarding in it: only someone who holds that a model must drink from the world, not only from other models.

Action

Keeps the well: holds back genuine human data, marks its provenance, and blends the real into every training run. Refuses to train on output alone, and pays to gather and label the true where the synthetic is free. Does the one thing that holds the tails open: keeps a source of the world outside the recursion.

Shadow

Not a saviour: real data is finite and slow and dear where synthetic is endless and instant and free, and the well cannot be kept by one hand alone. Because any builder gains by drinking the cheap pool while others pay to keep it clean, and marking provenance asks every publisher to label what they need not, the Wellkeeper spends against the abundance of the made and the free-rider's incentive at once, and the store of true data, once the commons is fouled, is a fixed and dwindling thing no diligence refills. It can hold the tails open, only by paying for the real that the free synthetic will always undercut, and only if others pay too.

Tell

Says "keep some of the world outside the loop, or the models will only ever echo each other." Is right, and being right means paying for the dear real while everyone else drinks free.

15 · place editorial

The Common.

Shown

The open web and the shared corpus, an ocean everyone draws from and pours into, growing by the day. Model output flowing in unmarked and settling among the genuine, the water reading clean while its share of echo climbs year on year.

Holds

The recursing, and the contamination it carries. Here every model's output returns to the water the next model will drink, so the Common holds the point where a generation's samples become a generation's training data: the made pours in and cannot be flagged, so it is where the loop closes, one model's narrowed world becoming the next one's source.

Offers

An ocean of data, free and endless. Because the Common is where all output collects, it offers every builder abundant cheap water without the cost of gathering the real, so it hands out the pool the budget loves, and hands out a pool ever more an echo of the models that fed it, so the abundance is genuine and the freshness is not.

Withheld

The mark that would tell the made from the real. The Common cannot flag what it never recorded, so it withholds the provenance that would let a builder drink only the genuine, and, being open to all and gating none, it withholds any brake on the recursion running through it, so the loop tightens unseen and the water grows flatter with every turn.

16 · place editorial

The Corpus.

Shown

The training set assembled for a build, a great body of text and image scraped from the common and handed to the model. Real and synthetic mixed through it unmarked, the tails already thinner than the world's, and the model learning it as if it were the world entire.

Holds

The generating, and the corpus's silent thinning. Here the model meets the data it will learn, so the Corpus holds the point where a drawn sample becomes a learned distribution: whatever is dense in it the model masters and whatever is sparse it half-learns, so it is where a corpus already poorer in the tails becomes a model poorer still, the narrowing passed from data to learner.

Offers

A world to learn, cheap and ready. Because the Corpus is the assembled data a model trains on, it offers the builder a whole distribution to fit without the cost of gathering it fresh, so it hands out the training set the schedule needs, and hands out one already part echo, its tails already shaved, so the model learns a world a shade narrower than the real.

Withheld

The rare it never held enough of. The Corpus cannot give the model what it barely contains, so it withholds the tails that were already thin in it, and, keeping no mark of provenance, it withholds any sign that its water is part echo, so the model learns the narrowed corpus as the whole world and cannot know what was missing.

17 · place editorial

The Vault.

Shown

A set-aside store of genuine human data, gathered, marked for provenance, and guarded from the recursion. The wellkeeper blending it into each run, and the pull against it: the free ocean outside, and every builder who would rather drink cheap than pay to keep the store.

Holds

The preserving, and the cost that resists it. Here the real is kept apart from the pool and fed deliberately into training, so the Vault holds the point where the remedy meets its price: a store of true data blended into every run holds the tails open, and it costs the dear and finite against the free and endless, so it is where keeping the world in the models is bought against the abundance of the made.

Offers

The world, kept outside the loop. Because the Vault is where genuine data is held and marked, it offers the one source that is not an echo, so it hands out the real that holds the tails open, and hands it out dear and slow and finite, so the fidelity it preserves is exactly as costly as the synthetic is cheap.

Withheld

The abundance and the ease of the free pool. The Vault cannot make the real as endless or as instant as the made, so it withholds the cheap plenty the cheap pool offers, and, a store that only dwindles once the commons is fouled and that one hand cannot keep alone, it withholds any promise that the well will hold if others drink free, so the remedy is real and dear and never safe from the free-rider.

19 · piece editorial

Model Collapse.

Place

Everywhere the production runs: from the corpus through the common down the generations, and against the vault where it is resisted. It is the whole arc, the learner undersampling the tails and the pool laundering the made until the distribution collapses to a self-referential mean.

Load bearing

Without it the other pieces are parts without a name: a shape, a lost edge, an echo, a store of the real, and no account of how one becomes the next. With it the misfit is whole: model collapse is the trap entire, a learner that must miss the rare and a pool that cannot mark the made summing, generation over generation, to models that forget the world and resemble only each other, the efficiency and the forgetting shown to be one thing. It is load bearing because it is the structure the whole production exists to stage (Shumailov et al.): the frame that binds the undersampled tail, the recursive pool, the compounding narrowing, and the finite real well into a single bend, so the piece carries the phenomenon as a whole, a field that feeds on its own output until the world is gone from it.

Apparent

Progress, more data and better models every year, an ecosystem thriving on its own abundance. Seen at the surface, model collapse looks like the opposite of itself, a field awash in cheap plentiful data and models growing more fluent, and the truth that the abundance is increasingly echo, that fluency on the common middle hides the loss of the rare, that a field training on its own output is quietly forgetting the world, stays hidden behind how healthy an ocean of cheap data and a fluent model appear.

Yearbook

The whole of it: Model Collapse is the model undersampling the tail of the distribution and pouring the made back through the pool, compounding down the generations, answered only by the real well kept dear against the free. It is the misfit named: a field feeding on its own output until it echoes only itself, no poisoner in it and no free hero, the world forgotten a shade at a time.

20 · piece editorial

The Distribution.

Place

Learned at the corpus, where the model fits its middle well and its tails barely. Held, in its genuine form, only in the vault, the store of the real kept outside the loop.

Load bearing

Without it there is nothing to narrow, and a model's samples are just samples, neither faithful nor forgetful. With it the whole trap has its measure: a real shape with a dense middle and sparse tails, so that a learner fitting finite samples captures the middle and misses the edges, and every reproduction is judged against the true shape it falls short of. The Distribution is load bearing because it is the standard the collapse is a departure from (statistical-learning theory): the tails are where the diversity of the world lives, so the piece carries the true shape whose rare edges the recursion strips away, the world the models are supposed to hold and steadily forget.

Apparent

An abstraction, a statistician's curve, a thing the model surely reproduces well enough. Seen at the surface, the distribution looks fully captured, the model fluent across it, and the truth that its sparse tails are undersampled and lost first, that the rare edges carrying the world's diversity are exactly what a finite learner cannot keep, stays hidden behind how complete a fluent model's coverage of the common middle appears.

Yearbook

Approximated by the model through generating, the true shape it fits in the middle and misses at the edges. It is the measure of the trap: the Distribution's sparse tail is what the model undersamples and the recursion forgets, its middle all that survives the collapse, and its genuine form is what only the real well keeps outside the loop.

21 · piece editorial

The Made.

Place

Poured into the common, where it mixes indistinguishably with the real. Drawn back at the corpus, where it becomes the next model's training data.

Load bearing

Without it the loop does not close, and a model's undersampling is a one-time shave with nowhere to compound. With it the recursion has its carrier: the model's output, poorer in the tails, returns to the pool and becomes training data, so the loss of one generation is fed to the next as fact. The Made is load bearing because it is what turns a single shave into a spiral (Shumailov et al.; Alemohammad et al.): synthetic data indistinguishable from the real and thinner in the tails is what each generation drinks, so the piece carries the model output whose unmarked return closes the loop and compounds the forgetting.

Apparent

Ordinary data, clean and useful, as good as any and cheaper to gather. Seen in the pool, the made looks like any other text or image, fluent and serviceable, and the truth that it is an echo already poorer in the tails, that training on it feeds the next model a narrowed world, that its very indistinguishability from the real is what launders the loss into the commons, stays hidden behind how genuine well-formed model output reads.

22 · piece editorial

The Real Well.

Place

Kept at the vault, held back and marked and blended into each run. Set against the common, the free fouled pool it is the costly alternative to.

Load bearing

Without it the play has no answer, and the recursion runs to a flat mean with nothing to hold the tails. With it the misfit has its costly cure: a store of genuine human data, marked for provenance and blended into every training run, is the one thing that keeps the world in the models, and it works only where the real is paid for against the free synthetic. The Real Well is load bearing because it is the remedy that carries its own scarcity (provenance and data-ecology work): the genuine is finite, slow, and dear where the made is endless and free, and once the commons is fouled the store only dwindles, so the piece carries both the cure for the collapse and the free-rider trap that undoes it.

Apparent

An easy fix, just use real data, keep some human text aside and be done. Seen hopefully, the real well looks like a simple safeguard, a reserve anyone can draw on, and the truth that genuine data is finite and dear where synthetic is free and endless, that marking provenance asks every publisher to label what they need not, and that any builder gains by drinking the cheap pool while others pay to keep the well, stays hidden behind how obvious keeping some real data sounds.

Yearbook

Kept by the wellkeeper through preserving, the store of the genuine held outside the loop. It is the cure the play cannot make painless: the Real Well blends the distribution back into training to hold the tail open against the made, and cannot do so without paying the dear finite real against the free endless synthetic, undone by any builder who drinks the cheap pool while others keep it clean.

23 · piece editorial

The Tail.

Place

Undersampled at the corpus, too seldom seen to be learned. Absent from the common a little more each generation, until it is gone from the water entirely.

Load bearing

Without it the collapse has no first casualty, and the narrowing is a vague loss with nothing specific to name. With it the mechanism has its point of entry: the rare event, seen too seldom for the learner to hold, is the first thing each generation drops, so the tails go before the variance and the variance before the shape. The Tail is load bearing because it is where the forgetting begins and where the world's diversity lives (Shumailov et al.): the rare is what a finite learner cannot keep and a recursion cannot recover, so the piece carries the sparse edge whose loss is the leading edge of the collapse, the first of the world to be forgotten.

Apparent

A negligible fraction, an edge case, a sliver the model can surely spare. Seen alone, the tail looks like a rounding error, a rare event of no great weight, and the truth that the tails hold the diversity and truth of the distribution, that losing them first is the collapse beginning and that once gone from the recursion they cannot be relearned, stays hidden behind how small a fraction the rare seems next to the dense common middle.

Yearbook

Dropped by the model through collapsing, the rare edge of the distribution too seldom seen to keep. It is the leading edge of the trap: the Tail is the first thing each generation forgets, the diversity the made lacks, the world's variety that only the real well, blended back, can hold open against the narrowing.

25 · position editorial

Model.

Has

A distribution to learn from finite samples, and the means to generate more like it. A sound competence: the common case mastered from many sightings, the rare one glimpsed too seldom to hold. No way to keep what it never saw enough of, only to reproduce what it learned.

Orders

Generate the likely. Learn the distribution you are shown and give it back, the everyday in full and the rare as best you can. Reproduce the world you saw, faithfully.

Loses

The tails, exactly by learning from finite samples. Because it fits the dense middle far better than the sparse edges, the seat cannot reproduce the rare it barely saw, so every batch of its samples is a shade poorer in the tails than its training data, and it has no power to keep a variance it was never shown enough of to learn. It is faithful to what it saw and, being faithful, gives back a world already a fraction narrower.

Drives

Toward a world reproduced a shade blander, faithful for one generation and forgetting across many, so the fidelity that serves each model is what carries the narrowing to the next.

26 · position editorial

Pool.

Has

A vast open body of data that everyone draws from and pours into. An openness that made it a commons: no gate, no toll, no ledger of where anything came from. No means to mark the made from the real, only to hold whatever is published.

Orders

Take it all in. Absorb every output, genuine and generated alike, and offer it back as one cheap endless ocean. Keep no provenance, ask no origin, gate nothing.

Loses

Its fidelity, exactly by the openness that makes it useful. Because it takes in all output and cannot mark the synthetic, the seat fills year on year with model echoes it cannot flag, so it cannot both stay the open commons everyone draws from and keep the water true, and its refusal to gate is what lets the recursion run unchecked through it. It grows ever larger and ever more an echo, laundering the made into the real by keeping no record of the difference.

Drives

Toward a commons that keeps no provenance, open to all and true to none, so the very openness that made it a shared good is what lets it drown the real in the made.

27 · position editorial

Trainer.

Has

A model to build, a budget and a schedule, and an ocean of cheap data at hand. A sound reason to use it: the pool is abundant and free, and gathering fresh human data is slow and dear. No cheap way to tell the made from the real in what it draws, only the choice to draw it.

Orders

Train on what's there. Take the abundant data at hand, machine-made and genuine alike, and feed it to the model. Use what is cheap and plentiful, the way any sensible builder would.

Loses

The freshness of its training data, exactly by drawing the cheap pool. Because the pool is already part model output and it cannot cheaply mark which, the seat trains on a corpus already thinned, and its own model's output pours back to thin it more, so it cannot both draw the abundant cheap water and keep the world in its data. It builds on schedule and under budget, and drinks an ocean that was already an echo.

Drives

Toward a model trained on the data at hand, sensible for one builder and narrowing for all, so the efficient use of what is there is the move that closes the loop on the tails.

28 · position editorial

Wellkeeper.

Has

A means to hold back genuine human data, to mark provenance, and to blend the real into every run. A clear case: a model that drinks only from other models forgets the world. No way to make the real as cheap as the synthetic, and no power to make other builders pay to keep the well.

Orders

Keep the well. Hold back genuine human data, mark what is real, and blend it into every training run so no model trains on output alone. Keep a source of the world outside the loop.

Loses

The savings and the ease, exactly what the cheap pool offers. Because real data is finite and slow and dear where synthetic is endless and free, and any builder gains by drinking the cheap pool while others keep it clean, the seat cannot hold the tails open without paying the price the pool waives and bearing the free-rider's undercut, and the store of true data, once the commons is fouled, only dwindles. It buys the world's presence in the models against the abundance of the made and the incentive of every builder to free-ride, and cannot hold it alone.

Drives

Toward a store of the real kept outside the loop, bought dear against the free synthetic and the free-rider, so the one seat that would keep the world in the models must pay for it while the rest drink free.

30 · process editorial

Collapsing.

Initiated by

A loss that feeds the loss, generation over generation. At the common, each model learning from a world narrowed by the last and narrowing it again, the shave compounding turn by turn.

Direction

Toward a distribution creeping to a flat self-referential mean. Each generation reproduces a world already poorer in the tails and drops a little more, so collapsing drives the sequence downward, the rare gone first, then the variance, then the shape itself, until the models resemble the world less than they resemble each other.

Lever

The compounding of a per-generation loss no single step would notice. Because each model's shave is a shade too small to mark, and each feeds the next, the losses accumulate unremarked, so collapsing runs on the invisibility of the single step, the forgetting assembled from narrowings none of which alone looked like harm.

Echo

Models that know the middle and have forgotten the edges. The output is fluent and self-consistent and thinner than the world by more each generation, the rare gone past recovering, so what reads as models steadily reproducing themselves is, in truth, a distribution collapsed to a bland average of an average.

31 · process editorial

Generating.

Initiated by

A model and a corpus to learn from. At the corpus, the learner fitting the dense middle from many sightings and the sparse tails from few, then producing samples of its own.

Direction

Toward samples faithful to the common and thin at the rare. The model fits what it saw often far better than what it saw seldom, so generating drives its output to reproduce the dense middle in full and the tails as thinly as it learned them, each batch a shade poorer in the rare than the data behind it.

Lever

The nature of learning from finite samples. Because a learner can only fit what it has seen enough of, and the rare is by definition seen too seldom, undersampling the tails is not an error but what generating from limited data must do, so the process runs on sound learning, the narrowing built into fidelity itself.

Echo

A model faithful to its training, and a shade blander than the world. The samples read as clean and true and reproduce the everyday exactly, and they carry a fraction less variance than the data that made them, so what reads as a model faithfully giving back what it learned is, unseen, the first shave off the tails of the world.

32 · process editorial

Preserving.

Initiated by

A commons seen to be filling with echoes, and someone unwilling to train on output alone. At the vault, the wellkeeper holding back real data, marking its provenance, and blending it into each run.

Direction

Toward a source of the world kept outside the loop. The wellkeeper sets the genuine apart and feeds it deliberately into training, so preserving drives real human data into every run against the pull of the cheap pool, holding the tails open by keeping the models drinking from more than each other.

Lever

The plain truth that a model drinking only from models forgets the world. Because output trained on output alone collapses, the case to blend the real is sound, so preserving runs on the same sense that built the models, turned now to the dear discipline of paying for the genuine where the synthetic is free.

Echo

The tails held open, at a price that only holds if others pay it. The store of real data has kept the world in this run and the recursion is checked where it is blended, and the real spent was dear and finite and the free pool still undercuts it, so what reads as a solved problem is, in truth, a well kept against every builder's incentive to drink cheap and let it dry.

33 · process editorial

Recursing.

Initiated by

A model's output published, and a builder needing data. At the common, the samples pouring back into the shared pool unmarked, and the next trainer drawing from it for the next build.

Direction

Toward each generation training on the last one's output. The model's samples flow into the pool and the next corpus is drawn from that pool, so recursing drives the output of one generation to become the training data of the next, the made mixed unmarked with the real and the loop closing turn by turn.

Lever

The abundance of the pool and the openness that keeps no mark. Because the cheap pool is what every builder sensibly draws and the commons keeps no provenance to warn them, each draught of already-thinned water feeds the next model, so recursing runs on efficiency and openness together, the loop closed by moves no one can fault.

Echo

A corpus drawn, and an echo passed on. The new model has its abundant cheap training data and the pool has taken in another round of output, and the water is a shade more echo and a shade less world, so what reads as reusing the plentiful data at hand is, unseen, the loop tightening on the tails.

35 · plan editorial

Generate the Likely.

Direction

Learn the distribution you are shown and give it back. Reproduce the common case in full and the rare one as best you can from the few times you saw it. Generate the likely, faithfully, the everyday mastered and the tails as thin as your finite view of them allows.

Orders

(The Model generates the likely and drops the rare, producing samples faithful to the dense middle and poorer at the sparse tails.)

Targets

  • [ ] the distribution is learned, the dense middle far better than the sparse tails
  • [ ] the likely is generated in full and the rare as thinly as it was seen
  • [ ] each batch of samples is a shade poorer in the tails than the training data
  • [ ] the loss is a shade no one built the model to keep, and none would notice in one generation

36 · plan editorial

Keep the Well.

Direction

Keep a source of the world outside the loop. Hold back genuine human data, mark its provenance so the real can be told from the made, and blend the real into every training run so no model trains on output alone. Pay for the true where the synthetic is free, and keep the tails open.

Orders

(The Wellkeeper holds back real data, marks provenance, and blends the genuine into every run, refusing to train on output alone.)

Implementation

(The Vault; the preserving process; the real well is held back and blended against the recursion, at the cost of the dear and finite over the free and endless.)

Targets

  • [ ] genuine human data is held back and its provenance marked
  • [ ] the real is blended into every run, so no model trains on output alone
  • [ ] the tails are held open against the narrowing
  • [ ] the cost is borne: the dear finite real against the free endless made, undone if others free-ride

37 · plan editorial

Take It All In.

Direction

Take in everything published, genuine and generated alike, and keep no mark of which is which. Grow with every output, gate nothing, ask no origin, and offer the whole back as one cheap endless ocean. Be the open commons, true to its openness if to nothing else.

Orders

(The Pool takes it all in, the made mixed unmarked with the real, and offers it back to the next builder as one undifferentiated ocean.)

Targets

  • [ ] all output is absorbed, genuine and generated alike
  • [ ] no provenance is kept, so the made and the real cannot be told apart
  • [ ] the commons grows ever richer in model output year on year
  • [ ] the next builder draws an ocean already an echo, and cannot see that it is

38 · plan editorial

Train on What's There.

Direction

Train the new model on the abundant data at hand. Take what is cheap and plentiful, machine-made and genuine alike, because the pool is free and endless where gathering fresh human data is slow and dear. Use what is there, the way any sensible builder would.

Orders

(The Trainer trains the new model on the data at hand, drawing the cheap abundant pool without sorting the made from the real.)

Targets

  • [ ] the abundant, cheap data at hand is drawn for training
  • [ ] the made and the real are taken alike, unsorted, because telling them apart is costly
  • [ ] the model is trained on a corpus already part model output
  • [ ] its own model's output will pour back to thin the pool further

39 · reference warrant

Line Of Work.

Staging model collapse as a learning system: a generative model that fits the dense middle of a distribution far better than its sparse tails, so each generation of its samples is poorer in the rare cases than the data it trained on, feeding an open data pool that cannot mark the synthetic from the real, so the next model trains on a corpus already thinned and narrows it again, until, down the generations, the tails are forgotten, the variance shrinks, and the distribution converges on a low, flat, self-referential mean. The play models the documented mechanics, the learner that undersamples the rare because it is seen too seldom, the synthetic output flowing back into the shared commons unmarked, the builder rationally drawing the abundant cheap data at hand, the compounding loss as output feeds the next run, the irreversible forgetting of the distribution's tails, and the store of genuine human data that alone holds the tails open, at a cost, as the load-bearing vectors of the structure, not as a tale of a careless engineer or a poisoned well. The governing finding of the source is the play's governing law: when generative models are trained recursively on data produced by earlier models, they progressively lose information about the true distribution, the tails first, and degenerate toward a narrow approximation that resembles reality ever less. It is distinct from the house's The Commons, where a rivalrous resource is depleted by extraction: there a finite stock is used up, here the data commons is not consumed but contaminated, its fidelity recursed away as output re-enters, the resource degraded rather than drained. It is distinct from the house's Overfitting, where a model fits one sample's noise and fails on new reality: there the fault is within a single fit on a fixed sample, here it is a multi-generation degeneration in which each model's honest output becomes the next one's training data. And it is distinct from the house's Bad Coin, where good money is hoarded and shoddy circulates at one bar: there the good is withdrawn by choice, here the real is drowned by recursion, no one hoarding and nothing withdrawn.

40 · reference warrant

Origin.

The phenomenon in practice, and the research spine beneath it.

Source Key Work / Event Scope
Shumailov et al. "The Curse of Recursion: Training on Generated Data Makes Models Forget" (arXiv, 2023); Nature (2024) The spine: recursive training on model output degrades the distribution, the tails vanishing first, converging on a degenerate mean. The play's governing law.
Alemohammad et al. "Self-Consuming Generative Models Go MAD" (2023): model autophagy disorder The autophagy: the self-consuming loop across generations, quality and diversity falling without a fresh supply of real data. The Forgetting.
Statistical-learning theory the bias-variance and finite-sample account of learning; estimation of distribution tails The mechanism: a learner fits the dense middle from finite samples and undersamples the sparse tails, so its samples are poorer in the rare. The Model.
Provenance / data ecology data-provenance and watermarking work; the "low-background steel" analogy for pre-model human data The remedy and its cost: telling the real from the made, and the finite, dwindling store of genuine human data once the commons is fouled. The Real Well.
Practitioner the open web filling with generative output; frontier-model training on scraped, increasingly synthetic corpora The trap in the field, and its cost: a shared data commons contaminated by its own output, held open only by preserving and blending real human data.
41 · reference warrant

Restrictions.

What the house refuses to model, and how it credits what it stages.

  • No villains: the elements are reduced to load-bearing vectors. The play refuses to cast the builder as careless or any party as poisoning the well, because the source finding is precisely that each move is sound: training on the abundant data at hand is what every builder sensibly does, machine-made data reads as well as any and costs far less to gather, and no one publishes model output meaning to wreck the commons. The collapse is authored by no poisoner and emerges from the structure, a learner that must undersample the rare and a pool that cannot mark the made from the real, so the misfit is that a field training on its own efficient, abundant output forgets the world it was meant to model.
  • No hero: the play refuses to resolve the misfit into a costless correction. The one move that holds the tails open is to keep the well, to hold back genuine human data, mark provenance, and blend the real into every run, and it asks the dear and finite where synthetic is free and endless, asks every publisher to label what they need not, and is undone by any one builder who drinks the cheap pool while others pay to keep it clean. The reckoning is a commons kept against the price of each builder and the abundance of the made at once, and a store of true data that, once the pool is fouled, only dwindles, so the cure for a field forgetting the world is a real-data well finite, dear, and easily free-ridden away.
  • Credit, not claim: the research above is other people's work, packaged and attributed here, never claimed as the house's. The staging is the house's; the model-collapse finding is Shumailov and colleagues', the self-consuming-model framing is Alemohammad and colleagues', and the finite-sample account of tail loss is the statistical-learning literature's, and the condition the trap actually requires, recursive training on model output through a pool that cannot mark the synthetic from the real, is carried as carefully as the finding itself.
42 · reference warrant

Encoding.

Source to constraint, per element.

  • The Distribution: carries the true shape of the data (statistical-learning theory), its sparse tails the diversity of the real that the learner undersamples.
  • The Tail: carries the rare cases (Shumailov et al.), lost first and then past recovering as the recursion runs.
  • The Made: carries the synthetic output (Shumailov et al.; Alemohammad et al.), the model-generated data flowing back into the pool unmarked.
  • The Real Well: carries the remedy and its scarcity (provenance and data-ecology work), the finite store of genuine human data that alone holds the tails open.
  • Model Collapse: carries the phenomenon whole (Shumailov et al.), a field training on its own output until it forgets the world and echoes only itself.

updated: 2026-07-03

43 · download artifact

Download Misfit.

Download model_collapse.zip
size
69.4 kB
sha256
2fe1ccfd5f5f86bfe75dc8f74e8284dc1bded5c3ad9aba9e6117bfe821a60799

Includes the misfit configuration and all staged elements.

Enter the Stage