Kimi K3 did not kill softmax. It accreted to it.

Open attention design rarely replaces the old layer. It accretes to it. Kimi K3, the open-weight model Moonshot AI shipped and SemiAnalysis credits with sweeping leaderboards at announcement, makes the pattern plain: it did not drop softmax, the standard mechanism transformers use to weigh which words to focus on. It kept softmax and added a new linear attention layer, Kimi Delta Attention, beside it. The hybrid stack is the lesson, not the replacement.

Linear attention is a faster, lower-memory alternative that historically traded recall quality for cost. Moonshot's contribution, decoded in SemiAnalysis's primer, is a hybrid: keep softmax where it works, route the rest through a compressed state the lineage traces through DeltaNet and Gated DeltaNet. Removing softmax inside that new layer drops per-step cost from quadratic in sequence length to linear. The catch is the compressed state has to be stabilized, which is what Kimi Delta Attention actually does.

The repeatable move: when an open lab ships a frontier result with a non-mainstream primitive, the real question is which layer it sits beside, not which layer it evicted. Most architecture stories tell the eviction. The mechanism underneath is the accretion.