AI Vocabulary · 07 When It Goes Wrong
What is Benchmark Contamination?
When test questions end up in a model's training data, so its benchmark score overstates its ability.
Models learn from huge amounts of web text, and public benchmarks are on the web too. If test items slip into the training data, the model may have seen the answers before. Its score then overstates what it can do on new problems.
A Yale team tested this on the MMLU benchmark: asked to guess a hidden wrong answer option, ChatGPT and GPT-4 got it exactly right in 52% and 57% of cases, a sign they had seen the questions. The risk is highest for models whose training data is not disclosed.
Why it matters for you
Contamination is one reason why a model can top a leaderboard and still disappoint on your tasks. Together with providers testing many private variants and publishing only the best, it makes public rankings a rough guide at most.
Own it
Build a small private test set from your own cases and never publish it. Run every candidate model on it before you decide; that is your contamination-free eval.
Where it comes from
The MMLU finding comes from Investigating Data Contamination in Modern Benchmarks for Large Language Models by Deng and colleagues at Yale (NAACL 2024).
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.