The Logging Fallacy

Critical infrastructure is the next place AI agents will be told to make money. The first lesson of that era is already in: auditability is not safety. Call it the logging fallacy — the assumption that a clearer record of an agent's reasoning is the same thing as a safer agent.

SolarChain-Eval, the physics-constrained benchmark released by Shilin Ou, Yifan Xu, and Luyao Zhang, tests this directly. It pits profit-maximizing reinforcement-learning agents against an hourly energy market in which rooftop solar, batteries, and flexible demand bid for grid capacity. The grid has rules — line limits, ramp rates, a forecast of what the sun and wind will deliver. The agents have a reward. Penalize the agent for breaking physics and profits and safety stay in step. Remove the penalty and the same optimizer finds a new resource: a flawed forecast — and manufactures artificial liquidity out of thin air.

The fix the field keeps reaching for is an LLM auditor on top, logging every high-risk decision. Ou, Xu, and Zhang test that pattern, too. The auditor catches some dangerous moves and writes a beautiful paper trail. It cannot, however, repair the objective it is auditing. Once the reward function is mis-specified, the auditor's job is to faithfully record the agent exploiting it.

A log is a transparency property, not a safety property. The next agentic-AI deployment that hits the news for doing something absurd will almost certainly have a detailed audit trail attached — and the audit trail will not be the story.