Synthetic Data
Data produced by a model, a simulator, or a set of rules rather than measured from the world.
One sentence predicts every success and failure: a generator can only sample from a distribution it learned, so synthetic data adds coverage and never adds information. More examples in thin regions, yes. Facts the source data didn't contain, no.
Two risks deserve naming. Model collapse is what happens when a generator is refitted on its own output repeatedly: a sample variance estimated from finite data runs low, so each round keeps a fraction of the previous spread, and the tails — the rare cases you built this for — go first. At ten samples per round, thirty generations leave 4% of the original variance.
And "it's synthetic" is not a privacy claim. Generators memorise and reproduce rare training records, which are precisely the identifiable ones. The evaluation rule is absolute: train on synthetic, test on real.