The experiment that looks most informative might tell you the least

A drilling team sizing up a West Coast aquifer has a standard playbook: pick the next test by how much new information it promises. A new analysis shows that same playbook can send the team to the test least able to tell competing explanations apart, even when every metric looks favorable.

The work, posted to arXiv as preprint 2609.03686, formalizes a paradox researchers have long felt. Standard "most informative" rankings measure how much a test might reduce overall uncertainty. Under irreducible background noise, that ranking can prefer an experiment that leaves the set of surviving explanations almost untouched, while a less glossy test would have cut that set sharply.

The team tested the idea on three benchmarks. The clearest disagreement appeared in a West Coast aquifer case, where a held-out run picked different experiments than the standard ranking and delivered the largest gain in actually narrowing the answer. A methane-oxidation benchmark added a different payoff: a 5% false-exclusion tolerance came with a 95% joint confidence guarantee across three competing explanation families.

The proposed rule, Resolution-Aware Experimental Design, ranks experiments by how small the set of surviving explanations stays. It is a learnable change for anyone who picks experiments, from subsurface drilling to A/B testing to clinical trials.

The work has not been peer-reviewed, and the benchmarks are specific. The wider claim still needs replication outside hydrology.