Researchers show a cheaper way to give AI long-term memory

Long conversations don't have to keep paying the full-history tax. A public preprint called TransMem shows a frozen language model can carry compressed notes from its own past turns and skip re-reading them on every reply.

Standard long-context systems re-encode the entire history at each turn, which is why long chats get expensive. TransMem instead harvests the model's own internal "hidden states" — the intermediate scratch work a network produces as it reads — and stores them as a small set of memory slots. A tiny learned gate then injects that memory back into the model's later layers without retraining the underlying network.

The method is plug-in: the backbone stays frozen, and only the memory block and gate are trained, using a self-distillation setup where a memory-augmented student matches a teacher that only sees the relevant evidence. Default config attaches a one-layer memory block to the last four layers with four memory slots each.

Reported numbers (all author-reported, not peer-reviewed): on the MemoryAgentBench long-memory benchmark, average accuracy rose from 29.54% to 40.00%. F1 gains measured 11.58 to 29.25 points on LoCoMo and 10.20 to 13.03 points on HotpotQA across model sizes. Code, training scripts, and checkpoints are public on GitHub, alongside the arXiv paper.

The honest limit: this is a preprint, the backbone still ingests the full context once to harvest those states, and no shipping product has been shown to use it.