70 text games, no manual: Inside DiG-bench, the benchmark that hides its rules to see if AI can find them

A new text benchmark called DiG-bench, short for Discovery in Games, withholds the rules of 70 handcrafted text games and asks models to figure them out by playing. Version 1 went up on arXiv on 12 August 2026, with an official leaderboard hosted at digbench.ai.

The benchmark was built by researchers from the University of Oxford, Princeton, KAUST, the Swiss AI Lab (IDSIA), Inria, and MIT. The games are all text-based, with action sets ranging from 2 to 34 choices per step, and organized into seven difficulty tiers. Twenty-one of the 70 games are public; the other 49 are kept private so models cannot train on the answers before evaluation. Humans solve all 70 on first attempt. Frontier models do not.

Import AI Issue 469 carries the most detailed public read of the leaderboard. On the basic track, with the model playing directly, Claude Opus 5 and Fable 5 with Claude Code lead overall, with GPT-5.5 close behind. Only Opus 5 and Fable 5 register any wins on the hardest tier, Tier 7, with a win rate of about 0.2, roughly a fifth of the games. With an agentic harness, Opus 5, GPT-5.5, and Kimi K3 also clear some Tier 6 games; GLM-5.2 and Gemini 3.1 Pro reach Tier 4. These numbers are a curator's read of the benchmark rather than a direct re-check of the tech report, so the leaderboard should be read as a first public snapshot, not a settled scoreboard.

The benchmark is built to measure a different axis than most AI evaluations. Standard tests of large language models test rule-following: hand the model a specification and check whether it does what the spec says. DiG-bench withholds the specification. Every game uses the same human and model interface, and the only way to make progress is to play, observe outcomes, and form a hypothesis about the rules that govern the environment. A model that scores well has to do more than pattern-match against training data: it has to run small experiments on the game itself and update its model of the world as it goes. That is what the benchmark means by rule-discovery.

Import AI 469 folds DiG-bench into a broader "science AI" thread, paired with two adjacent releases: Paradigm Research's RSI Simulator, which models recursive self-improvement in a controlled environment, and Inherent's Faraday and Replica AI-scientist systems. The underlying question, in all three, is whether AI systems can drive scientific or technical work on their own. The second-order effect of the leaderboard is harder to pin down, but the direction is clear: a benchmark that measures rule-discovery from play, on a held-out set, gives a public signal for whether the underlying capability is moving at all, not just whether a particular model was tuned against a particular exam.

The takeaway is not that AI is "getting smarter" in some general sense. It is that a concrete, public, mostly held-out benchmark now exists for a specific capability: inferring the rules of a new environment from play. Humans do this in early childhood. The best frontier systems, by the leaderboard Import AI reports, do it about a fifth of the time on the hardest tier. The next public snapshot of the leaderboard, as the team expands the held-out set, is the watch item.