AIBRIEF
The new arXiv method reads a model's internal geometry to flag contamination, even after post training has scrubbed the behavioral evidence.
When an AI model tops a public leaderboard, did it learn the skill or just memorize the test? Detecting that benchmark contamination, where test items leak into training data and inflate scores, has been hard for two years, especially after a model goes through reinforcement-learning post-training, the stage where the behavioral trail prior detectors relied on gets scrubbed.
A new method called Excess Separability, posted to arXiv on August 13 and accepted at MAPR 2026 in Hue City, Vietnam, reads a model's internal activation geometry to close that gap. Author Florian Braun's approach needs no training-data access and no retrained reference model, the two practical blockers that have kept audit-style contamination detection out of most workflows.
A 2024 NAACL study by New, Marone, and Van Durme found roughly 29% of MMLU questions already appear in Common Crawl, and Yang et al. (2023) showed that paraphrased items can produce scores that exceed genuine capability. Production fixes already exist, including post-contest test suites like LiveCodeBench, repo-based tests like SWE-bench Pro, and Humanity's Last Exam in Nature, but each trades recency or scope for difficulty.
Excess Separability does not solve contamination. It solves the specific configuration where a model has been post-trained past the point where behavioral detectors can see the cheating. The signal moves inside the model.