The Execution-Fidelity Gap

The execution-fidelity gap is the new front line for any LLM agent that has to act on the physical world. The pattern: an agent and an operator can agree a plan is feasible, and the physics can still disagree by a wide margin. Treat the agent's self-report as the answer, and the system inherits the gap.

A controlled benchmark from arXiv 2608.04265 demonstrates the shape. The authors built a simulated smart-grid demand-response system, with 40 prosumers (homes and small sites that both consume and produce power) on a radial feeder (a tree-shaped power-distribution line), and let LLM planning agents run the schedules. They compared four executor architectures: predefined, sequential, hierarchical, and a brute-force search. The forced-search executor was the oracle across every baseline seed. The other three architectures left measurable value on the table.

The portable rubric, built from this single demonstrative case, is three questions to run before any plan goes live. Does the executor architecture match the problem? Will the plan finish before the deadline? Is feasibility treated as a probability rather than as a binary self-report? Apply a deadline-feasibility gate and the benchmark's regret metric (the gap between the chosen plan and the best feasible plan) drops 61.1% below the fixed-sequential baseline. Skip the gate, and the deployment inherits the 2.68x voltage-shortfall multiplier the arXiv 2608.04265 benchmark measured when it let the agent's stated plan stand without a real feasibility check. The grid is the case study. The rubric is built from one demonstrative case; cross-domain portability is a first-principles inference, not a demonstrated property. Robots, autonomous logistics, building systems, transit, and manufacturing face the same test.