Memory Persistence Cuts Token Use Up to 71.7% in Multi-Agent Systems

When an AI agent handles the same class of support ticket ten times, it should get faster and cheaper at it. In practice, most don't. They reset to zero after every session — which is fine for a demo, catastrophic for production. A paper posted to arXiv on March 27 attempts to put numbers on exactly how bad this gets, and what memory architecture actually fixes it.

The paper is Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systems (arXiv:2604.03295), by researchers at Emory University, Illinois Institute of Technology, the University of Notre Dame, and Cisco Research. The key finding worth pulling apart: a memory system that persists experience across sessions reduced token usage by 9.4 percent to 71.7 percent compared to memoryless baselines, while improving task accuracy on MultiAgentBench's coding, research, and database environments.

The most striking number in the paper is not the accuracy delta — it's the cost. The full-context baseline, which keeps every previous interaction in the prompt, hits 72.9 percent accuracy on the LOCOMO long-horizon memory benchmark. It also takes 9.87 seconds median latency per conversation and burns through roughly 26,000 tokens per session. Mem0's state of AI agent memory report calls this "the only approach that is categorically unusable in real-time production settings." The accuracy leaderboard and the production-viable leaderboard are not the same list.

LLMA-Mem, the framework proposed in the paper, takes a different approach. Instead of dumping everything into context, it maintains three distinct memory layers: episodic memory records what happened, procedural memory stores distilled strategies that worked, and transactive memory tracks which agents on a team are good at what. When a task comes in, the system retrieves relevant context from the appropriate layer rather than scrolling through a full history log.

The architecture also lets teams pick a memory topology. Local topology gives each agent its own private memory store — useful when agents shouldn't share context. Shared topology gives the whole team a common store — good for tight coordination. Hybrid topology splits the difference: local episodic memory with shared higher-level structures. The paper shows these choices matter. In some task distributions, shared memory wins. In others, local or hybrid wins by a significant margin. No single topology dominates across all environments.

This connects directly to the non-monotonic scaling finding that makes the paper worth reading beyond its numbers. The researchers find that increasing team size does not produce monotonic improvements in long-horizon performance. Larger teams can outperform smaller ones, but only when memory architecture supports reuse of experience. When it doesn't, you get teams of eight agents each starting from scratch — which is eight times the cost of one agent, with no compounding benefit from accumulated experience.

The implication for the current wave of multi-agent frameworks — AutoGen, CrewAI, LangGraph, and the rest — is straightforward. The dominant answer to "how do I make my agent system smarter?" is currently "add more agents." The paper argues the better answer might be "give the agents you have a memory system that actually remembers."

The researchers make their code available at github.com/ShanglinWu/MAS_lifelong_learning, including a Python package with a documented quickstart. The package installs via pip and ships with a local demo that doesn't require API keys — which is the right way to release research infrastructure. Researchers and practitioners can inspect the retrieval logic, the consolidation pipeline, and the topology configuration without negotiating an API quota first.

Caveats worth noting: the MultiAgentBench evaluation covers three environment categories; production agent deployments often run in environments the benchmark doesn't map to cleanly. The token reduction range spans 9.4 percent to 71.7 percent, and the paper doesn't fully explain what determines where a given deployment falls within it. The LOCOMO benchmark — the evaluation dataset for long-horizon memory — is relatively new, so the comparison baselines are still being established. And as always with an arXiv preprint, these results have not yet undergone peer review.

What the paper does well is name the actual tradeoff that production engineers face and has been dealing with quietly for the past two years: more agents or better memory. The industry has mostly chosen more agents. This paper is a reason to revisit that choice.