How Geometry-Based Data Splits Skew AI Benchmarks

A model can win a benchmark leaderboard and still underperform in production. Engineers typically blame distribution shift, model drift, or a flaky deployment pipeline. A new arXiv preprint argues the gap can start earlier, in how the benchmark's data was sliced in the first place.

The paper, Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods, runs head-to-head tests of five common ways to divide a dataset into a training portion and a held-out test portion. Two are familiar: random splitting, which assigns rows to either set by coin flip, and stratified sampling, which preserves class balance across both sets. Three are geometry-based methods more common in chemometrics, where they were designed to spread calibration samples evenly across an input range: Kennard-Stone, Duplex, and SPXY, each of which picks splits based on the geometric distance between data points rather than at random.

The authors measure how similar each resulting train and test set is, using three statistical tests: the chi-square test, which checks whether two categorical distributions match; the Kolmogorov-Smirnov test, which compares two empirical distributions; and Maximum Mean Discrepancy, or MMD, a kernel-based test for whether two samples could plausibly come from the same underlying distribution.

Across fifteen UCI Machine Learning Repository datasets, the geometry-based methods produce near-zero MMD scores. The two halves of the data are statistically almost indistinguishable, which sounds like a feature. The authors argue that this level of similarity is itself a problem: when the test set mirrors the training set too closely, downstream performance estimates become unstable, and the model that an AutoML system "wins" on starts to depend on artifacts of the split rather than the underlying signal.

AutoML, or automated machine learning, refers to systems that pick, tune, and ensemble models with minimal human input. A core promise of AutoML is that its leaderboards generalize: the model that scores highest on a benchmark should keep scoring highest on new data. The preprint, available in full on arXiv, suggests that when splits are too clean, the leaderboard ranking itself is brittle. Similarity between train and test is necessary for a fair evaluation, but it is not sufficient, and pushing similarity too high can be its own form of bias.

The constructive response the authors offer is an Optimised-Distribution splitting method that treats similarity as an explicit optimization target. Rather than picking points by geometric distance, the new method searches for a split whose train and test sets fall just inside a target similarity band: similar enough to be a fair test, different enough that the leaderboard reflects real generalization pressure. The new method reports an 89.0% mean MMD similarity across the strategies evaluated, against the near-zero scores of the geometry-based baselines. That is a real number, but its scope is narrow: fifteen datasets, all from the UCI repository, a benchmark suite widely seen as a useful but aging reference. Modern AutoML evaluation practice also runs on broader suites like OpenML-CC18 and TabZilla, and the preprint does not show the new method winning outside UCI.

An aggregator summary of the paper frames the work as a step toward more honest AutoML evaluation. That framing is fair up to a point. The deeper takeaway is that the splitting algorithm is itself a load-bearing variable in any reported model performance, not a neutral preprocessing choice. Two papers that report the same score on the same dataset can be measuring two different things if they used different splits, and a reader who treats a benchmark number as a stable claim is reading past the methodology section.

The preprint is not peer-reviewed, the 89.0% figure is an author-reported mean rather than an independently replicated result, and the comparison is limited to similarity metrics rather than downstream accuracy gains. The next check is whether the proposed splitting method holds up on OpenML-CC18 and TabZilla, and whether similarity-tuned splits actually translate into more stable leaderboards when a real AutoML pipeline runs end to end.