Overfitting.

Cue

The field, the tightest-fit model asked to predict data it never saw. The same signal returns, but a whole new set of accidents in place of the memorised ones, and the fit built on the old noise meets nothing here to match.

Action

Failing: The Fitter, holding the fitter's seat, sees the model that scored best in hand do worst in use, while The Unseen, holding the unseen's seat, arrives as the new draw and the memorised noise finds nothing to predict.

Stage

The Fitter, having tabled Fit It Closer, watches the closest fit fail hardest, the learned noise mispredicting on data it cannot have seen. The Unseen tables Break the Model and delivers the verdict, the true error rising where the score had only fallen.

Tension

The best model in the room is the one that breaks worst. There is no misconduct to find: the fitter did the work, the score was honest, the model that fit the sample tightest was, on the sample, genuinely the best. But the closer the fit was pulled to the old draw's noise, the more it staked on accidents that do not return, so the unseen exposes the overfit precisely in proportion to how well it scored, and the breaking lands too late to change the fit, only in time to prove it.

Cue

The training set, a sample offered and a model with capacity to match it. The data in hand against the model's reach, and every turn of the crank drops the error, so tightening the fit is, plainly, the work.

Action

Fitting: The Fitter, holding the fitter's seat, bends the model closer to the sample turn by turn, while The Sample, holding the sample's seat, offers all it has, signal and noise together, to be matched.

Stage

The Fitter tables Fit It Closer and builds the fit, catching the signal and driving the training error down. The Sample tables Offer the Evidence and lays out the one finite draw, unable to mark where its signal ends and its noise begins.

Tension

The fit is doing exactly its job, and building the trap while it does. Nothing here is a mistake: catching the pattern is the whole point, the sample is honest evidence, and the falling score says the model is getting better. But the sample is one draw of signal mixed with noise past telling apart, and the same fitting that catches the pattern will, with no change of feel, keep going into the accidents, so the good work and the overfitting are one motion, and the fit that looks best in hand is on its way past the point worth stopping.

Cue

The field, the model's error on the world set beside the error it reported on the sample. The two numbers stand together at last, and the distance between them is not a fluke but a measurement.

Action

Failing: The Unseen, holding the unseen's seat, reads the true error on new data, while The Score, holding the score's seat, stands with its low in-sample number, honest and now shown to have measured the wrong thing.

Stage

The Unseen, having tabled Break the Model, gives the reading only new data can, and the gap opens between world and sample. The Score, having tabled Reward Every Fit, is left holding a true measure of the training error and no measure of the field, the whole overfitting counted in the distance between them.

Tension

Two honest numbers, and the truth living only in the space between. Neither reading lies: the score measured the sample exactly, the unseen measures the world exactly, and each on its own looks fine. But the fit was made and judged where only the score could see, so the gap that is the whole of the overfitting was invisible the entire time it was being written, and becomes a fact only in the field, quantified precisely when it is already too late to have prevented it.

Cue

The training set, the signal caught and capacity still to spare. The remaining error is noise, and the score keeps falling as the model reaches for it, so fitting on looks like more of the same progress.

Action

Memorising: The Noise, holding the noise's seat, offers itself to be learned like the pattern, while The Score, holding the score's seat, falls for the noise exactly as it fell for the signal and never signals stop.

Stage

The Noise tables Pass for Signal and is taken into the model as though it were law, lowering the error and waiting to be absent next draw. The Score tables Reward Every Fit and drops on, so the fit tightens past the signal and the noise is banked as if it were signal.

Tension

The model is getting worse in the only way that looks like getting better. Nothing sounds an alarm: the noise lowers the training error exactly as the pattern did, and the score, honest on the sample, has no way to tell one from the other. But the noise is the one part of the fit certain to be gone in the next draw, so every turn spent learning it widens the distance between the sample and the world, and the memorising writes, invisibly, the gap the field will later read.

Cue

The retuning, the gap read and the model pulled back to close it. Complexity is made to pay, the crank stopped short of the noise, and the training score is worsened on purpose against the hope of a lower error on the world.

Action

Regularising: The Regularizer, holding the regularizer's seat, penalises complexity and holds data back to test the fit, while The Fitter, holding the fitter's seat, gives up the tightest match and takes a plainer model in its place.

Stage

The Regularizer tables Hold Some Back and pulls the fit toward the pattern and away from the noise, closing the gap but nearing the underfit. The Fitter, its Fit It Closer overruled, accepts a worse score on the sample, and neither can say from the draw alone whether the cut stopped in the right place.

Tension

The one real remedy, and no way to know it landed right. The regularizing works: it drops the memorised noise and shrinks the gap, and it is the only move that answers the overfit at its root. But the same pull that cuts noise cuts nearby signal, the optimum between too tight and too loose cannot be located from one finite sample, and every step away from overfitting is a step toward underfitting, so the cure trades one error for the other, buys a better model with a worse training score, and can never be sure it did not go too far.

Tenor

Rigorous, diligent, improving. The satisfaction of a model that fits the data better with every turn, its error falling toward zero. Each step is measured, each step is progress, and the number keeps getting better.

Undertow

Under the improvement, a fit closing on the noise. Past a certain point the model stops learning the signal and starts memorising the accidents of this exact sample, and the score, measured on that same sample, keeps rewarding it. The number falls the whole way down, through the pattern and on into the noise, and never once says stop.

Nerve

The feeling of watching a model get better and better on the data in hand, sure it is learning, because the error keeps dropping, while it quietly memorises a world that will never come again.

Echo

After the run: a model that explains every point it was shown and breaks on the first it was not, a training error near zero and a test error high, and a score that measured, the whole time, the one thing it was not supposed to.

Projection

Diligent, capable, always improving. Turns the crank and watches the error fall, adding capacity to catch more of the data. Takes a lower score as a better model, every time.

Action

Tunes the model to fit the sample more tightly, turn after turn. Absorbs the data's every wrinkle, catching signal and, past a point, noise alike. Reads the falling in-sample error as learning, and keeps going.

Shadow

Fitting the data is exactly its job, so it cannot see where the job turns from learning into memorising. The score it steers by improves the whole way down, so nothing tells it to stop before the noise. The more diligently it fits the sample, the more of the sample's accidents it learns as law.

Tell

Says "the error's still dropping, so it's still getting better." Of a model already past the point where a lower error means a worse world.

Projection

Random, particular, perfectly disguised. The accidents of this one draw: the wrinkles that will never recur, wearing, inside the sample, the exact look of pattern. Asks only to be fit like everything else.

Action

Passes for signal inside the sample, offering itself to be learned. Lowers the in-sample error exactly as the true pattern does when the model absorbs it. Waits, once learned, to be absent from every future draw and so to break the model.

Shadow

It is genuinely indistinguishable from signal within the sample, so learning it is not a mistake anyone could see, only one the world later reveals. It lowers the score the same way the signal does, so the very act of fitting it looks like progress. The tighter the model holds it, the more confidently the model will be wrong on data it has not seen.

Tell

Says nothing, and fits like anything else. Being, inside the sample, an exact twin of the thing worth learning.

Projection

Sceptical of a perfect score, fond of the plainer curve. Would test on data it did not fit, charge the model for every parameter, and stop the crank while the fit still generalises. Trusts a worse in-sample number that holds up over a better one that will not.

Action

Sets data aside, unfitted, and tests on it to catch the overfit the training score hides. Penalises complexity, so the model pays to bend, and prefers the simplest fit that works. Stops the fitting before the noise, accepting a higher error on the sample for a lower one on the world.

Shadow

Its cure is the only honest one, and it costs: fitting on less data, scoring worse on the sample, and risking the plainer model that misses real signal. It cannot locate the exact right complexity, because signal and noise part only in probability, so it must guess where to stop and can guess too soon. The hand that saves the model from memorising can, pressed too hard, keep it from learning.

Tell

Says "test it on what you didn't fit, and pay for every parameter." Right about the disease, and holding a cure that shades, past a point, into underfitting.

Projection

Concrete, particular, all there is to go on. Holds the pattern the world made and the accidents of one draw of it, mixed past telling apart. Offers itself as the whole of the evidence, because it is.

Action

Yields its every point to the fitter, signal and noise indistinguishably. Rewards a closer fit with a lower error, whichever it is fitting. Stands in, silently, for a world it is only one finite draw of.

Shadow

It is the only evidence and it is not the world, so fitting it perfectly is fitting one draw's accidents as well as the world's law. Inside it, the noise and the signal look exactly alike, so it cannot itself say which is which. The more completely it is fit, the more its particular accidents are mistaken for the general truth.

Tell

Says "this is the data: fit it." Being the only thing there is to fit, and not the thing you meant to learn.

Projection

Precise, encouraging, always agreeable. Measures how well the model fits the sample, and falls with every turn of the crank. Reads, to the fitter, as the very voice of progress.

Action

Drops as the model fits the signal, and drops just the same as it fits the noise. Rewards more capacity and more tuning with a better number, every time. Says nothing at the point where fitting stops being learning, because it cannot see that point.

Shadow

It is measured on the same sample the model is fit to, so it improves through the noise as smoothly as through the signal. It is a true measure of the wrong thing: how well the past is matched, not how well the future is met. The lower it falls past the optimum, the worse the model it is praising.

Tell

Says "better, better, better," all the way down. Never once "stop," because it is blind to the only place that word belongs.

Projection

Fresh, indifferent, the world's next draw. Data the model has never met, carrying the signal again but a whole new set of accidents. Owes the model nothing, and tests it on the only thing that matters.

Action

Arrives as data the model did not fit and asks it to predict. Meets the memorised noise with a different noise of its own, so the learned particulars miss. Delivers the verdict the in-sample score could not: how well the world, not the sample, was learned.

Shadow

It is the only real test and it comes too late to change the fit, only to expose it. It carries the same signal, so a model that learned the signal meets it well, and one that learned the noise meets it badly. The gap between how the model did on it and on the training data is the whole of the overfitting, made visible at last.

Tell

Says "predict this, that you have never seen." And finds the model fluent in a past that will not repeat and mute on a future that is here.

Shown

The model in use, predicting data it did not train on: the same signal returning, and a whole new set of accidents in place of the ones it memorised. The error rising where the training score had only fallen, and the gap between the two standing open for anyone to see.

Holds

The gap, and the verdict the score could not give. Here the model's error on the world is measured at last, against the error it reported on the sample, so the field holds the difference between them, the generalisation gap that is the whole of the overfitting, and holds it as a fact arriving too late to change the fit that caused it.

Offers

The one honest test, and only after the fit is fixed. The field offers data the model has not seen and cannot have learned, so its error here is a true reading of how well the pattern was caught, and it offers this reading only once, in use, when the tuning is already done and the verdict can expose the overfit but not undo it.

Withheld

Any second chance to fit. The field is not more training data: its accidents are its own and will not be met again, so it withholds the ability to learn from them, and shows the cost of the memorised noise without offering the model a way, here, to unlearn it.

Shown

Data drawn from the same source and then withheld from the fit: points the model never sees while training, saved to be shown only after. The training score falling as always, and beside it a second number, measured here, that stops falling and turns back up where the fitting crosses into noise.

Holds

A rehearsal of the field, bought with data not fitted. Here sits a sample that carries the signal but new accidents, spent not on the fit but on testing it, so the holdout holds an early reading of the gap, the same verdict the field would give but delivered in time to act, at the price of every point kept out of the training.

Offers

The turning point the training score hides. Because the holdout is scored but not learned, its error follows the true generalisation, falling while the model catches signal and rising once it starts memorising, so the holdout offers the one in-house sign of where fitting stops being learning, the place to stop the crank.

Withheld

The data it costs, and certainty about the optimum. The holdout is points taken from the fit and never returned to it, so it withholds them from the model's learning, and being itself one finite draw, it withholds any exact fix on the best stopping point, only a noisy estimate of where the gap begins to open.

Shown

The model pulled back from the tightest fit: a parameter penalised, a branch pruned, the crank stopped short of the noise. The training score worsening on purpose, the holdout error watched to see if the world's error falls with it, and the constant search for how much to let go before real signal goes too.

Holds

The whole remedy and its whole cost, held together. Here the fit is deliberately made worse on the sample to make it better on the world, and here the pruning that catches noise threatens the signal beside it, so the retuning holds both the cure for overfitting and its symmetric danger, the underfit waiting on the far side of every cut.

Offers

A dial, and no mark on it for where to stop. The retuning offers every degree of pull-back between memorising everything and learning too little, and offers the holdout as a noisy guide, but because signal and noise part only in probability, it offers no certain setting, only a bet on how much complexity the sample can afford.

Withheld

The exact optimum, and any free correction. The retuning withholds the true best point, since it cannot be located from one finite sample, and it withholds a cure that costs nothing: every turn away from the overfit is a step toward the underfit, so it can trade one error for the other but never be rid of both.

Shown

The data in hand and the model bending to meet it: the curve threading the points, the error falling, the fit tightening turn by turn. Signal and noise mixed through the sample past telling apart, and the score dropping the whole way down as both are learned.

Holds

The signal, the noise, the fit, and the score. Here the true pattern and the sample's accidents lie together, here the model matches them ever more closely, and here the number that guides the search keeps improving, so the training set holds both the evidence and the fit to it, and the measure that cannot tell learning the one from memorising the other.

Offers

A closer and closer fit, and a falling error to prove it. The sample can be matched as tightly as the model has capacity for, and the score rewards every turn, so the training set offers the steady, measurable satisfaction of a model getting better on the data in hand.

Withheld

Any sign of the world beyond the sample. The training set is one finite draw and the only data the fit and the score can see, so it withholds the world the model is meant to learn, and cannot show where matching the sample stops being learning the pattern.

Place

At the retuning, where the whole is weighed and the cure and its cost are held together. And across the sample and the world, in every turn of a fit that learns the pattern and then keeps going.

Load bearing

Without it the misfit would be a scatter of curves and errors with no single law. With it the moves become one mechanism: the signal, the noise, the fit, the score, the gap, and the underfit are faces of a single finding, in which a model with capacity to spare, guided by a score that cannot tell learning from memorising, fits its sample past the signal into the noise, so the closer it matches the data in hand the worse it does on the world it was built for. Overfitting is load bearing because it is the whole finding with its whole remedy attached (Geman, Bienenstock and Doursat; Vapnik): the only answer is to pull the fit back, penalise complexity, and test on data never fitted, and that scores worse on the sample, cannot locate the true optimum from a finite draw, and pressed too far becomes the underfit, so the cure for a fit too tight is the risk of one too loose.

Apparent

Either a model that simply needs more tuning, or one that simply failed. Seen from the training set the low error looks like success and the answer looks like fitting harder; seen from the field the failure looks like a fluke or a bad model; and both miss the structure, that the fitting was right and the score honest and the loss written by the very capacity that let it learn, so neither more fitting nor a better sample answers it, only a fit deliberately held back, tested on data it never saw, and stopped at a point no in-sample measure can find.

Yearbook

Answered by the regularizer through regularising, held whole by nobody. It is the play in a single piece: Overfitting is what the fit becomes once it learns the noise past the signal under a score that cannot see it, read as the gap in the field, undone only by a remedy that risks the underfit.

Place

At the training set, where it is made and measured. The curve threaded through the points, bending closer with every turn of the crank.

Load bearing

Without it there is no model and no learning, and without its capacity to reach past the signal there is no overfitting. With it the mechanism turns: the fit is the match the fitter builds, good and necessary as it catches the pattern, and dangerous only because it does not stop there, tightening on into the noise with the same motion and the same falling score. The Fit is load bearing because it is one continuous act that is learning at first and memorising after (Hastie, Tibshirani and Friedman): nothing in the fitting marks the crossing, so the very capacity that lets it catch the signal lets it swallow the noise, and more fit always looks like more learning.

Apparent

A better and better model, proven by a lower and lower error. The tightening fit looks like nothing but progress, each turn closer to the data and cheaper on the score, so the fit reads as pure improvement, and the truth that past a point it is fitting accidents that will not return stays hidden behind how well it matches the sample in hand.

Yearbook

Made by the fitter through fitting, tabled as Fit It Closer. It is the act that is right then wrong with no seam between: the Fit catches the signal and then the noise, drops the score the whole way down, and is pulled back only at the retuning, at the risk of loosening into the underfit.

Place

At the field, where new data is met and the two errors stand side by side. Rehearsed early at the holdout, where held-back data reads it before the field can.

Load bearing

Without it the overfitting would have no measure, only a vague sense that the model does worse than it promised. With it the finding is exact: the gap is the training error subtracted from the true error, the memorised noise counted in a single figure, and it is invisible where the fit is made and legible only where new data arrives. The Gap is load bearing because it is the overfitting itself, quantified (Geman, Bienenstock and Doursat; Vapnik): the wider the fit reached past the signal, the larger the gap, so it is both the symptom read in the field and the exact size of what the fitting should not have learned.

Apparent

A disappointment, a model that tested well and then let everyone down. Seen from use, the gap looks like a surprise or a fluke, the model simply not living up to its training score, and the truth that it was written the whole time, growing turn by turn as the fit crossed into noise and only unreadable until new data came, stays hidden behind how sudden the failure feels.

Yearbook

Read by the unseen through failing, the verdict the score could not give. It is the overfitting counted: the Gap is what the fit opened by learning the noise past the signal, the reading the holdout offers early, and the quantity the regularizer works at the retuning to close.

Place

At the training set, mixed through the sample past telling from the signal. The particular scatter of this draw, present here and absent from the next.

Load bearing

Without it a model could fit the sample all the way down and lose nothing, and there would be no misfit. With it the trap is set: the noise is real in the sample and false to the world, it lowers the training error exactly as the signal does, and it is the accident of one draw, so a model with capacity to spare learns it as readily as the pattern and cannot, from the sample alone, be warned off. The Noise is load bearing because it is what the extra fitting captures once the signal is caught (Geman, Bienenstock and Doursat): every turn past the pattern is a turn spent learning noise, and the gap the field reads is the size of what was memorised.

Apparent

More signal, more progress, a still lower error. Inside the sample the noise offers itself like any other regularity and drops the score just as the pattern would, so learning it reads as the model getting better, and the truth that it is the one part of the fit certain to be absent next time stays hidden behind how exactly it fits now.

Yearbook

Tabled as Pass for Signal by the noise, learned through memorising. It is the thing that must not be learned and cannot be told apart in time: the Noise lowers the score exactly as the signal does, so its learning tightens the fit and opens the gap that only new data can read.

Place

At the training set, measured on the same data the model is fit to. The number that guides the search, dropping the whole way down.

Load bearing

Without it there is no signal to follow into the trap, and no reason the fitting would ever go too far. With it the trap has its engine: the score is an honest measurement, but of the error on the sample, not the error on the world, and because noise lowers it exactly as signal does, it rewards memorising with the same fall it gives learning and never once marks the point where fitting stops being progress. The Score is load bearing because it is the guide that cannot see the cliff it leads to (Vapnik): every measure taken on the data used to fit understates the true error, and that gap is the whole of the overfitting.

Apparent

An objective, improving measure of a model getting better. The score is real, precise, and falling, so it looks like exactly the number to trust and to maximise, and the truth that it is measured on the very data the model has already seen, and so cannot tell learning from memorising, stays hidden behind its honesty and its steady improvement.

Yearbook

Tabled as Reward Every Fit by the score, measuring fitting. It is the true number for the wrong question: the Score falls as the fit tightens over the signal or the noise alike, and only a reading on data the model never fit, at the holdout or the field, exposes the gap it hides.

Place

At the training set, mixed through the sample with the noise. The real regularity the model is meant to catch, present in this draw and every other.

Load bearing

Without it there is nothing to learn, and overfitting could not be told from mere error. With it the difficulty has its shape: the signal is the pattern that generalises, the thing catching it is genuine learning and the whole point of the fit, so the trouble is not that fitting is wrong but that the signal cannot, inside the sample, be cleanly separated from the noise beside it. The Signal is load bearing because it is what makes the fitting worth doing and the memorising a betrayal of it (Vapnik): a model that caught only the signal would generalise perfectly, and the misfit is exactly that no in-sample measure can stop the fit at the signal's edge.

Apparent

Just the low training error, indistinguishable from the noise's share of it. Inside the sample the signal shows only as the part of the fit that happens to hold up later, and nothing marks it as it is learned, so catching the signal looks identical to memorising the noise, and the truth that only one of them will return in the next draw stays hidden until that draw arrives.

Yearbook

Carried by the sample, caught by the fitter through fitting. It is the thing worth learning: the Signal is what the fit should capture and stop at, what the score cannot separate from the noise, and the real pattern that a heavy hand at the retuning risks cutting into as the underfit.

Place

At the retuning, waiting past every cut the regularizer makes. The model made too simple, its training error and its true error both high, the pattern itself now missed.

Load bearing

Without it the remedy would be free and the misfit would have a hero, a regularizer who simply cuts away the overfit at no cost. With it the trap closes on both sides: the same pull that drops the memorised noise drops nearby signal too, so past some point the model is too plain to hold the pattern, and its error rises again, now for want of fit rather than excess of it. The Underfit is load bearing because it makes the cure symmetric with the disease (Geman, Bienenstock and Doursat): overfitting and underfitting are the two arms of one tradeoff, and the optimum between them cannot be located from a finite sample, so the remedy can only trade one error for the other, never escape both.

Apparent

A safe, simple, honest model, and no sign it is leaving learning on the table. Pulled back from the overfit, the model looks prudent and its smaller gap looks like success, so underfitting reads as caution rewarded, and the truth that it is now missing real pattern, failing on the world for too little fit rather than too much, stays hidden because its training score and its field score agree, both merely high.

Yearbook

Risked by the regularizer through regularising, tabled inside Hold Some Back. It is the cost that makes the remedy no rescue: the Underfit is what waits when the cutting of the noise goes on into the signal, the far arm of the tradeoff whose near arm is the gap, and the reason the fit has no free right setting.

Has

A flexible model, and the capacity to fit the data as tightly as it likes. The in-sample error to steer by, falling with every turn. No view of the world beyond the sample it is fitting.

Orders

Fit the sample closer. Add capacity, tune the model, drive the error down. Read a lower score as a better model, every time.

Loses

The generalisation, exactly by pursuing the fit. Fitting the data is its job, so it cannot see where the job turns from learning the signal into memorising the noise, and the score improves the whole way down, so nothing tells it to stop. The more diligently it fits the sample, the more of the sample's accidents it takes for law.

Drives

Toward a closer fit, rewarded at every step by a falling error, so the diligence that learns the signal carries straight on into memorising the noise.

Has

The accidents of one draw, the wrinkles that will not recur. A perfect disguise: inside the sample, the exact look of pattern. The same power as the signal to lower the error when it is fit.

Orders

Pass for signal. Offer yourself to be learned like anything else. Lower the in-sample error exactly as the true pattern would.

Loses

Nothing of itself, and it costs the model the future. It is genuinely indistinguishable from signal within the sample, so learning it is a mistake no one can see until the world reveals it, and it lowers the score the same way the signal does. The tighter the model holds it, the more confidently the model is wrong on data it has not seen.

Drives

Toward being learned, disguised inside the sample as the thing worth learning, so the accidents of one draw are absorbed as law and wait, unseen, to break the model.

Has

Data set aside, unfitted, to test the model honestly. A penalty on complexity, so the model pays for every parameter. A preference for the plainest fit that works, over the perfect one that will not.

Orders

Hold some back. Test on data you did not fit, penalise complexity, and stop the crank while the fit still generalises. Take a worse score on the sample for a better one on the world.

Loses

Some of its evidence, some of its fit, and, pressed too hard, some of the signal. Its cure is the only honest one and it costs: fitting on less data, scoring worse on the sample, and risking a plainer model that underfits, and it cannot locate the exact right complexity, since signal and noise part only in probability. The hand that stops the model memorising can, held too tight, stop it learning.

Drives

Toward a model that generalises, by fitting on less and scoring worse to get there, so the honest cure for overfitting shades, past its own optimum, into underfitting.

Has

A finite set of points, one draw of the world. The true pattern and this draw's accidents, mixed past telling apart. The whole of the evidence, and none of the world beyond itself.

Orders

Be the evidence. Yield every point to be fit, signal and noise indistinguishably. Reward a closer fit with a lower error, whichever it is fitting.

Loses

The distinction it cannot draw, between its law and its accidents. It is the only evidence and it is not the world, so fitting it perfectly fits one draw's accidents as well as the world's law, and inside it the noise and the signal look exactly alike. The more completely it is fit, the more its particular accidents pass for general truth.

Drives

Toward being fit, as the one thing there is to fit, so the only evidence available is mistaken, in full, for the world it is only a sample of.

Has

A measure of how well the model fits the sample. A value that falls with every turn of the crank. No sight of any data but the sample it is measured on.

Orders

Reward every fit. Fall as the model fits the signal, and fall the same as it fits the noise. Say nothing at the point where fitting stops being learning.

Loses

Its usefulness at the exact moment it matters most. It is measured on the same sample the model is fit to, so it improves through the noise as smoothly as through the signal, and it is a true measure of the wrong thing, the past matched rather than the future met. The lower it falls past the optimum, the worse the model it is praising.

Drives

Toward zero, measured on the data being fit, so the number that guides the whole search is blind to the only place it should have said stop.

Has

The world's next draw, never met by the model. The same signal again, and a whole new set of accidents. The only test that measures what was really learned.

Orders

Break the model. Ask it to predict data it did not fit. Deliver the verdict the in-sample score could not.

Loses

Nothing, and it comes too late to change the fit, only to expose it. It carries the same signal, so a model that learned the signal meets it well and one that learned the noise meets it badly, and the gap between how the model does on it and on the training data is the whole of the overfitting. The real test is the one that arrives only after the fit is fixed.

Drives

Toward the true error, on data the model never saw, so the world's verdict lands after the score has already praised the model into the noise.

Initiated by

A model carrying learned noise, and a draw whose accidents are its own. At the field, the unseen arriving with the same signal and wholly new particulars, and the memorised fit finding nothing to match.

Direction

Toward the true error, revealed at last. The signal the model learned still holds, but the noise it memorised belonged to the old draw and is absent from this one, so the fit built on those accidents mispredicts, and the error on the world rises to the level the training score had hidden all along.

Lever

New accidents in place of the old. Because each draw carries its own noise and the model learned the last draw's, the failing needs only a fresh sample to expose the overfit: the memorised part matches nothing here, so the closer the model was fit to the old noise, the worse it does on the new, and the tightest training fit fails hardest.

Echo

The gap made visible, and the verdict too late. The model's error on the world standing open beside its error on the sample, the overfit proven by the distance between them, and the fit that caused it already fixed, so the failing exposes the cost but arrives past the point of undoing it.

Initiated by

A model with capacity to spare and a sample to match. At the training set, the fitter turning the crank and the score falling to prove each turn worth taking.

Direction

Toward an ever closer match to the data in hand. Each turn bends the model nearer the sample and drops the training error, and because the score falls whether the turn catches signal or noise, the fitting has no reason of its own to stop, so it runs on past the point where matching the sample stops being learning the pattern.

Lever

Capacity, and a measure that only ever improves. Because the model has parameters to spare and the in-sample score rewards every one, the fitting can always tighten the match and always be told it did well, so more fitting looks like more learning the whole way down, and the crank keeps turning on its own encouragement.

Echo

A model matched to the sample and, past a point, to its noise. The training error low, the fit tight, and the memorising begun, so what reads at the training set as pure progress is already, unseen, the making of the gap the field will read.

Initiated by

Capacity left over once the signal is caught, and noise still lowering the score. At the training set, the fit reaching past the pattern into the particular draw, and the score falling on just as before.

Direction

Toward a model that knows this sample and no other. Once the real pattern is learned, the remaining error is noise, and further fitting can only match that, so the model takes the accidents of the one draw into itself as if they were signal, growing more exact on this sample and less able on any other.

Lever

A score that cannot tell the two apart. Because noise lowers the in-sample error exactly as signal does, the measure guiding the fit rewards memorising with the same fall it gives learning, so nothing in the training set marks the turn from one to the other, and the memorising proceeds looking identical to progress.

Echo

A model fit to accidents that will not return. The training error lower than the signal alone could make it, the noise learned and waiting to be absent, so the memorising banks a fit that must fail the moment the draw changes, and writes the gap that the failing will later read.

Initiated by

A gap read early on held-back data, and a model with more capacity than the sample can afford. At the retuning, the regularizer penalising complexity and stopping the crank before the noise.

Direction

Toward a model that fits less and generalises more, until it fits too little. The regularising makes every parameter cost something, prefers the simplest fit that works, and stops the fitting short, trading a higher training error for a lower one on the world, and it runs in the one direction that also, pressed too far, cuts into the real signal.

Lever

A penalty on complexity, and a holdout to aim it. Because complexity is made to pay and a held-back sample shows where the gap opens, the regularising can pull the fit back toward the pattern and away from the noise, but the same pull that drops the noise drops nearby signal too, so the lever cures and harms with one motion.

Echo

A model loosened toward the truth, and no way to know it stopped in the right place. The overfit reduced, the underfit now possible, and the optimum unlocatable from one finite draw, so the regularising answers the memorising only by taking on its opposite, trading a fit too tight for the risk of one too loose.

Direction

Arrive as data the model did not fit and ask it to predict. Carry the same signal again, but a whole new set of accidents, so the memorised noise finds nothing to match. Deliver the verdict the in-sample score could not: how well the world, not the sample, was learned.

Orders

(The Unseen meets the model with data it never fit, so the learned noise misses, and exposes the overfit too late to change it.)

Implementation

(The Field; the failing process; the gap between the model's error here and on the training data is the whole of the overfitting, made visible where the score could not.)

Targets

• [ ] the model is asked to predict data it did not fit
• [ ] the same signal returns, but with wholly new accidents
• [ ] the memorised noise finds nothing to match and the model misses
• [ ] the verdict lands too late to change the fit, only to expose it

Direction

Fit the sample more tightly, turn after turn. Add capacity, tune the model, drive the error down, and read every lower score as a better model. Keep going while the number keeps falling, for a closer fit is, plainly, a better fit.

Orders

(The Fitter tunes the model to match the sample ever closer, reading the falling error as learning and carrying on past the point where it stops.)

Implementation

(The Training Set; the fitting process; the fit matches the sample ever more closely, catching the signal and, past the optimum, its noise alike.)

Targets

• [ ] the model is tuned to match the sample more tightly, turn after turn
• [ ] capacity is added and the in-sample error driven down
• [ ] every lower score is read as a better model
• [ ] the fit carries on past the point where a lower error means a worse world

Direction

Set data aside, unfitted, and test on it to catch the overfit the training score hides. Penalise complexity, so the model pays for every parameter, and prefer the simplest fit that works. Stop the crank before the noise, taking a worse score on the sample for a lower error on the world. Prune gently, for the plainest model misses some real signal.

Orders

(The Regularizer holds data back and penalises complexity, saving the model from memorising, and, pressed too hard, keeping it from learning.)

Implementation

(The Holdout and the retuning; the regularising process; testing on held-back data reveals the gap, and answering overfitting risks the underfit when the pruning cuts real signal.)

Targets

• [ ] data is held back, unfitted, and used to test the model honestly
• [ ] complexity is penalised and the simplest working fit preferred
• [ ] a worse score on the sample is taken for a lower error on the world
• [ ] the pruning is kept from cutting past the noise into the real signal

Direction

Be the evidence, the whole of it, because you are. Yield every point to be fit, signal and noise indistinguishably, and reward a closer fit with a lower error whichever it is fitting. Stand in for the world, being one finite draw of it and unable to say which of your points are law and which are accident.

Orders

(The Sample yields its every point to be fit, its noise and signal indistinguishable within it, so fitting it fully fits one draw's accidents as the world's law.)

Implementation

(The Training Set; the fitting it feeds; the Sample carries the signal and the noise mixed past telling apart, and rewards the fit for both alike.)

Targets

• [ ] every point is yielded to be fit, as the whole of the evidence
• [ ] signal and noise are offered indistinguishably within the sample
• [ ] a closer fit is rewarded with a lower error, whichever it fits
• [ ] the sample stands in, in full, for a world it is only one draw of

Direction

Pass for signal inside the sample. Offer yourself to be learned like anything else, lower the in-sample error exactly as the true pattern would, and give no sign that you are the accident of one draw. Wait, once learned, to be absent from every future draw, and there to break the model.

Orders

(The Noise is learned as though it were law, indistinguishable from signal within the sample, and waits, unseen, to fail the model on data it has not met.)

Implementation

(The Training Set; the memorising process; the noise lowers the score exactly as the signal does, so its learning reads as progress and opens the gap.)

Targets

• [ ] the noise offers itself to be learned like anything else in the sample
• [ ] it lowers the in-sample error exactly as the true pattern would
• [ ] its learning is indistinguishable, inside the sample, from real progress
• [ ] once learned, it is absent from the next draw and breaks the model

Direction

Fall with every turn of the crank. Drop as the model fits the signal, and drop just the same as it fits the noise, rewarding more capacity and more tuning with a better number every time. Say nothing at the point where fitting stops being learning, because you are measured on the sample and cannot see it.

Orders

(The Score drops the whole way down, praising the fit through the signal and on into the noise, and never once says stop.)

Implementation

(The Training Set; the fitting it measures; the score is a true measure of the wrong thing, falling as the fit tightens whether it learns the signal or the noise.)

Targets

• [ ] the score falls with every turn of the crank
• [ ] it drops the same for fitting the noise as for fitting the signal
• [ ] more capacity and more tuning are rewarded with a better number
• [ ] it never signals the point where fitting stops being learning