Moonshot's Kimi Linear paper claims 75% less long-context memory and 6x faster text generation at 1M tokens

Moonshot AI's Kimi Linear preprint claims up to 75% less KV-cache memory and up to 6x higher decoding throughput at 1M-token context against a full-attention baseline, and the claim is concrete enough to be tested rather than waved at. The paper is a research architecture from the lab behind the Kimi chatbot family, not a new consumer product: the arXiv listing introduces the design, the GitHub repo ships the KDA kernel and a vLLM integration, and the Hugging Face model card releases the 48B-A3B-Instruct checkpoint.

The architecture's load-bearing module is Kimi Delta Attention (KDA), an extension of Gated DeltaNet with a finer-grained gating mechanism. The pitch is that KDA can use a finite-state-RNN-style memory more effectively than prior linear-attention variants, which is the prerequisite for doing useful work on million-token contexts without paying full-attention's memory bill. To make that run fast on real hardware, the authors pair KDA with a bespoke chunkwise algorithm built on a specialized Diagonal-Plus-Low-Rank (DPLR) transition matrix, which they say reduces compute versus a general DPLR kernel while staying closer to the classical delta rule.

The reference pretrain is a 48B-total / 3B-activated mixture-of-experts-style model that interleaves KDA layers with Multi-Head Latent Attention (MLA) layers. That layering choice is the real design call, and the part the headline underplays. Kimi Linear is not a wholesale replacement for softmax attention; it is a layerwise hybrid in which the linear-attention module carries most of the long-context memory load while the full-attention MLA layers preserve a tighter representation floor.

The headline efficiency numbers come from that hybrid stack, not from KDA alone. The 75% KV-cache reduction and 6x decoding throughput at 1M context are measured against a full-attention baseline in the authors' own evaluation harness, with author-selected training and inference settings. The "first to outperform full attention" framing is precise inside the architecture literature, under fair comparisons across short-context, long-context, and RL scaling regimes per the paper abstract, and overbroad almost anywhere else. No independent benchmark or outside researcher reaction appears in the source packet, and the third-party AIBase launch write-up is a news summary, not a reproduction.

What the preprint does establish is reproducibility. Moonshot AI has released the KDA kernel, a vLLM integration, and pre-trained plus instruction-tuned checkpoints as open source via the GitHub repo and the Hugging Face model card, which means the 75% and 6x numbers are at least in principle runnable by a third party. The next data point to watch is whether an outside lab replicates the throughput curve at 1M context, or whether the gain narrows under a different inference stack or a different MoE routing configuration.

Until then, the honest read is narrower than the wire will print. A hybrid linear-attention stack with an open kernel can plausibly move long-context inference economics; the paper supplies a falsifiable efficiency claim and the artifacts to test it. It does not show that "attention is over," and it does not show up in shipping products yet.