Long-horizon AI agents lose track mid-task. A sidecar 'memory agent' that picks when to interrupt beats every alternative tested.

As trajectories lengthen, AI agents lose track of the requirements, environment facts, and prior diagnoses that defined the work in the first place. The arXiv preprint "Remember When It Matters" calls that failure "behavioral state decay" and argues the fix is not more memory or longer context, but a separate, smaller agent that watches the main one and decides when to interrupt with a grounded reminder.

The system puts two agents side by side. The action agent runs unmodified, with whichever frontier model a lab already uses and the harness that ships with it. A second agent, called the memory agent, inspects each turn, updates a structured memory bank, and decides whether to inject a single memory-grounded reminder into the action agent's next prompt or stay silent. Because the action policy is not touched, the module drops in front of Claude, GPT, or a local Qwen checkpoint without retraining anything downstream. That plug-and-play property is what makes the result portable rather than a single lab's lucky configuration.

The reported payoff is consistent across the two target benchmarks. On Terminal-Bench 2.0, a CLI benchmark for long-horizon coding agents maintained by the Harbor Framework project, the memory agent lifted pass@1 by 8.3 percentage points. On τ²-Bench, a dual-control conversational agent environment maintained by Amazon AGI, the lift was 6.8 percentage points. Both gains held across weaker and stronger action agents, not just the configuration the architecture was tuned for, which is the harder result to fake.

The selective memory agent beat all four ablation baselines tested: passive memory bank exposure, always-on injection of reminders at every step, advisor-only guidance that suggests a plan without writing to memory, and general retrieval. Memory for AI agents, the authors argue, is an active intervention problem, not a retrieval problem. In the runs, always-on injection was the obvious failure mode: low-signal reminders crowded the context window and hurt performance. The mechanism question the paper puts at the center is when to remind, not what to store.

To make the mechanism concrete, the authors trained a Qwen3.5-27B memory policy with supervised fine-tuning plus GRPO on a synthetic set of long-horizon trajectories called SETA. They describe the checkpoint as an "early step": validation reward improves and gains partially transfer to Terminal-Bench, but the release is meant as a starting point for further work. The +8.3-point lift is author-reported and not yet independently replicated, and the benchmark versions are documented through secondary mirrors (τ²-Bench on AISBench, Terminal-Bench 2.0 leaderboard).

The paper does not claim retrieval-augmented systems are broken as a class, only that on Terminal-Bench 2.0 and τ²-Bench an explicit decision about when to intervene beats all four ablation baselines, with the largest gaps on weaker action agents. What to watch next is whether SETA and the Qwen3.5-27B memory-policy weights become publicly available for replication, and how the Terminal-Bench 2.0 leaderboard treats the architecture once other labs can plug it in. A name for the underlying failure is starting to circulate.