A Smaller Language Model Outperformed a Bigger One at Helping AI Agents Navigate Blind Spots

AI agents that navigate by partial observation, like a robot in an unexplored room or a game-playing system acting on incomplete information, have long been promised help from small language models serving as onboard consultants. A new preprint finds that in vanilla setups, that help almost never arrives. The language model silently defers to the agent it was meant to correct, and the consultation window becomes a formality.

The paper, titled "ASK in the Dark: Uncertainty-Gated LLM Assistance under Partial Observability," measures the failure directly. Across the paper's three reinforcement-learning benchmarks, the consulting language model overrode the agent's own policy at or near zero percent of the time. The mechanism designed to call in outside judgment had collapsed into a redundancy check.

The setting is a partially observable Markov decision process, or POMDP, a standard formalism for decision-making under incomplete information. The agent sees what is in front of it and must commit to an action; the full state of the world stays hidden. The design pattern the paper targets is called uncertainty-gated assistance. The agent consults the language model only when its own action probabilities look spread out, a condition called predictive entropy.

A bare egocentric prompt, one that simply asks the language model whether the agent's intended action is reasonable, does not give the model enough to reason independently. Without knowing where the agent has already been or what it has tried, the model defaults to endorsing the policy it was asked to evaluate. That, the authors argue, is the structural reason the gating collapses: there is nothing for the language model to disagree with, since it cannot see what the agent has seen. The diagnosis is explicit: a context problem, not a capacity problem.

The decision to consult is made on every step using the agent's predictive entropy, the spread of its action probabilities. Under partial observability, that signal tracks action uncertainty rather than state uncertainty. The paper argues this matters: even when an agent is confidently wrong about the world, its action distribution can still register enough doubt to flag a moment where outside reasoning might help.

The fix, which the authors call ASK+, supplies the model with the partially revealed map, the positions the agent has visited, and the action history. Structured chain-of-thought reasoning then asks the model to talk through what it knows before issuing a verdict. With that context, the consulting model shifts from a passive rubber-stamp to an occasional active corrector of the agent's plan.

A 2-billionparameter language model given the trajectory-aware prompt and structured reasoning matched or beat a 4-billionparameter version of itself across the three test environments: DoorKey, FourRooms, and HigherLower. On DoorKey, success rate rose from 89 percent to 93 percent with ASK+. In FourRooms, success rate climbed from roughly 53 percent to 70 percent. A bigger model alone did not close the gap.

That result does more than flip a benchmark. It identifies an architectural lever: in this corner of agent design, careful prompt and context construction can substitute for raw parameter count. The authors have released an official codebase that should let other teams reproduce the finding and stress-test the prompt design on harder environments.

Two caveats bound the claim. The result is scoped to the three test environments studied, and the paper makes no claim about real-world deployments or larger POMDPs. The predictive-entropy signal used to decide when to consult the model also measures action uncertainty rather than state uncertainty. Under partial observability, the authors argue, action entropy remains informative even when the agent's beliefs about the world are wrong, but that asymmetry will limit where the technique transfers cleanly.

The next test is whether trajectory-aware prompting generalizes beyond navigation-style benchmarks, and whether larger POMDPs expose cases where state uncertainty, not action uncertainty, is what an agent most needs help with.