Data Concepts

Dataset Decontamination

If a benchmark's questions are in your training data, its score measures memory. At web scale they are, and checking is a search problem rather than a choice.

Eight percent of a benchmark leaking into the training corpus is enough to move a reported score from 0.62 to 0.649, because the contaminated slice is answered from memory rather than from ability

Why Does This Exist?

Every public benchmark is on the internet. The questions, usually the answers, and very often a blog post working through them.

If you trained on a web scrape, you trained on some of them. Not as a mistake — as an unavoidable consequence of the corpus, unless somebody explicitly looked and removed them.

And a contaminated benchmark doesn't fail loudly. It reports a higher score, which is the same reason leakage is the hardest bug class in this band: the failure mode is a better number. The published figure then measures a mixture of ability and recall, in unknown proportions, and no amount of staring at it separates the two.

So decontamination isn't a nicety at the end of a pipeline. It's the thing that decides whether any number you report means anything.

Think of It Like This

A student who has seen the paper

A candidate sits an exam and scores 94%. Excellent, until you learn that three of the twenty questions were on a past paper they revised from last week.

Here's what makes it awkward rather than simply cheating. They didn't steal the paper. Those three questions have been circulating in study guides for years, and any diligent student would have met them. Nobody did anything wrong, and the 94% still isn't a measurement of what they can do.

Worse, you can't correct it after the fact. You don't know which three, and you can't ask, because they can't reliably tell you either — a well-revised question and a genuinely understood one feel identical from the inside.

The only fix is upstream: check the paper against the study guides before the exam, and write fresh questions where they overlap.

How It Actually Works

What counts as contamination

More things than people check for.

Verbatim overlap — the question text appears in training. Answer-only overlap — the question doesn't appear but the answer does, alongside enough context to key on. Paraphrase — a rewording, which a hash-based check misses entirely. Solution walkthroughs — a tutorial that explains the reasoning, which is worse than the answer alone because it teaches the pattern. And derivative datasets — someone built an instruction-tuning set from a benchmark, you trained on the instruction set, and the original never appears anywhere in your provenance.

That last one is the sneakiest, and it's the reason a provenance-only argument ("we never downloaded that benchmark") isn't a decontamination claim.

Detecting it

The machinery is the same as deduplication, pointed at a specific target set.

N-gram overlap is the standard: build the set of n-grams from each benchmark item, and flag any training document sharing more than a threshold. Common settings are 8- to 13-gram overlap, and the choice of $n$ is the entire sensitivity dial — too small and every document matches on common phrasing, too large and a two-word paraphrase escapes.

MinHash and LSH scale it to a full corpus without comparing every pair, exactly as in deduplication.

Embedding similarity catches paraphrase, which n-grams cannot, at the cost of a much fuzzier threshold and a lot more false positives to adjudicate.

Canary strings are the preventive trick: benchmark authors embed a unique random string in their files, so a model that can reproduce the canary has demonstrably seen the file. Cheap, definitive when it fires, and it only works if the author thought of it.

What to do when you find it

Remove from training if you catch it before the run. That's the clean answer and it needs the check to happen before training rather than after.

Remove from evaluation if you catch it afterwards, and report the clean-subset score alongside the full one. This is the honest move available post-hoc, and it's rarer in write-ups than it should be.

Report the overlap rate. A benchmark score without a contamination figure is incomplete, in the same way an estimate without an interval is.

Keep a private evaluation set that has never been published anywhere. This is the only durable defence, because a public benchmark's contamination risk only ever increases. A few hundred carefully-written internal items, never posted, tell you more about a model over time than any leaderboard.

Why it keeps happening

The incentives point the wrong way. Decontamination lowers your reported number, costs engineering time, and produces nothing anyone celebrates. Nobody's promotion depends on discovering that a headline result was three points softer than published.

Which is why it works better as a pipeline gate than as a virtue: run the overlap check automatically against every evaluation set you use, fail the run if the rate exceeds a threshold you set in advance, and log the figure with the result. Same discipline as data validation — an assertion rather than an intention.

Worked example

A 1,000-question benchmark. The model genuinely answers unseen questions at 0.62. On contaminated questions it answers at 0.98, because it's recalling rather than reasoning.

At 8% contamination the reported score is $0.92 \times 0.62 + 0.08 \times 0.98 = 0.649$ . That's 2.9 points of inflation from 80 leaked questions — comfortably enough to change a ranking, and small enough that nobody suspects anything.

At 30% contamination, which is not unusual for an older benchmark, the reported score is 0.728. That's 10.8 points, and it would read as a substantial modelling advance.

The uncomfortable part: without measuring contamination, 0.649 and 0.62 are indistinguishable from the outside, and so are 0.728 and 0.62. The inflation is a function of a number nobody reported.

Show Me the Code

1import numpy as np2
3clean: float = 0.62       # accuracy on questions the model has never seen4memorised: float = 0.98   # accuracy on questions that were in the training data5
6def reported(clean_acc: float, memorised_acc: float, contaminated: float) -> float:7    """What a benchmark reports when part of it leaked into training."""8    return float((1 - contaminated) * clean_acc + contaminated * memorised_acc)9
10for rate in (0.00, 0.08, 0.30):11    print(rate, round(reported(clean, memorised, rate), 4))12# -> 0.0 0.6213# -> 0.08 0.648814# -> 0.3 0.72815print(round(reported(clean, memorised, 0.30) - clean, 4))  # -> 0.108

The function is linear in the contamination rate, which is the useful takeaway: inflation scales directly with overlap, so measuring the overlap tells you the size of the correction.

Watch Out For

Checking provenance instead of content

"We never downloaded that benchmark, so we're not contaminated." It sounds like a complete answer and it checks the wrong thing.

The benchmark is on the web, so it's in the scrape. Its questions appear in blog posts, Stack Overflow answers, GitHub repositories of solutions, and lecture slides. And derivative datasets are everywhere: an instruction-tuning set assembled from benchmark items carries no marker saying so, so you can train on the benchmark through three degrees of separation with a clean download log throughout.

Provenance tells you what you intended to include. Contamination is about what's actually in the bytes.

Run the content check: n-gram overlap between every evaluation set you use and your final training corpus, at a threshold you pick in advance, with the rate logged. It's the same MinHash and LSH infrastructure you already built for deduplication, pointed at a much smaller target set, so it's cheap once that exists.

An n-gram threshold that catches nothing, or everything

You run a 13-gram overlap check and it flags 0.1% of documents. Clean, apparently.

Thirteen consecutive tokens is a long exact match. Change two words, reformat a number, translate and translate back, and the overlap vanishes while the model still learned the item. So a high threshold reports a comfortable figure and misses paraphrase entirely — which is most of the real contamination, because most of the internet's copies of a benchmark are reworded.

Drop to 5-grams and the opposite happens: every document containing "what is the capital of" matches something, the flag rate hits double digits, and you can't act on it.

Do both. Run a tight n-gram check for verbatim overlap and an embedding-similarity pass for paraphrase, and adjudicate a sample of what each one flags by reading it. Then report both rates rather than the friendlier one. And keep a private evaluation set, because it's the only thing that stays uncontaminated by construction.

The Quick Version

Every public benchmark is on the web, so a web-scraped corpus contains some of it unless somebody looked.
Contamination doesn't fail loudly. It raises your score, which is why nobody notices.
It comes in five shapes: verbatim, answer-only, paraphrase, solution walkthroughs, and derivative datasets.
Provenance is not a decontamination claim. Check the content of the corpus, not the download log.
N-gram overlap at 8 to 13 tokens is the standard check, scaled with MinHash and LSH. Embedding similarity catches paraphrase.
Canary strings prove contamination definitively when the benchmark author included one.
Remove from training if you catch it early, remove from evaluation if you catch it late, and report the overlap rate either way.
8% contamination moves 0.62 to 0.649. 30% moves it to 0.728. Inflation is linear in the overlap rate.
A private, never-published evaluation set is the only durable defence.
Make it a pipeline gate with a threshold, because the incentives will never make it a virtue.