Kimi K3 reroutes the AI memory trade. It doesn't shrink it.

When Moonshot AI released Kimi K3, a 2.8-trillion-parameter Chinese language model, memory-chip stocks sold off on a familiar fear: that the model's new attention design would slash demand for the high-bandwidth memory and networking silicon that have defined the AI build-out. SemiAnalysis, one of the most-cited independent research shops covering AI infrastructure, argues the read is backwards. The same architecture choice that compresses one kind of memory traffic reroutes the demand onto another, and the rerouted version is the more expensive one.

The panic imported the DeepSeek R1 playbook from January 2025. R1's efficiency shock hit memory-chip names on the assumption that cheaper inference meant fewer chips. Six months later the trade was back, and the bears are running the same script on K3. SemiAnalysis's thread is the live counter-thesis hitting the tape.

Here is the mechanism, in plain terms. Transformers keep "context" in a structure called a KV cache, a short-term memory bank that records what the model has read so far. The cache lives in high-bandwidth memory, or HBM, the stacked DRAM that sits on the same package as the GPU. Longer contexts, longer documents, and longer conversations all push more traffic through HBM. K3 replaces that cache in three of every four layers with a design called Kimi Delta Linear Attention, or KDA, that compresses the cache. Per-layer KV-cache transfer drops by up to 10x versus comparable full-attention models. The bear read: 10x less HBM. The bull read, which is SemiAnalysis's read: HBM demand is set by total parameter count and active experts, not by per-layer cache shape.

The numbers explain why. K3 has 2.8 trillion parameters. Even at 4-bit precision, a single forward pass through one "scale-up domain," a tightly coupled cluster of GPUs acting as one logical machine, needs roughly 1.5 TB of HBM capacity and bandwidth. That is the floor, and KDA does not lower it. Linear attention shrinks the cache that moves between layers. It does not shrink the weights that have to be loaded for every token. The total memory footprint is set by the model, not by the attention design.

Then comes the part the bears missed. K3 is a mixture-of-experts model with 896 experts spread across the GPU pool, a layout called Wide Expert Parallelism, or WideEP. Each GPU holds only a small slice of the experts. For any given token, the system has to fetch the right expert from wherever it lives in the rack. The bottleneck is no longer the cache. It is the network that shuffles tokens and experts between GPUs at every step.

That is where the rack-scale trade shows up. NVIDIA's NVL72 systems, 72 GPUs wired into a single domain, ship with a copper backplane that delivers roughly 18x the bandwidth of comparable DGX B200 systems built from eight-GPU baseboards. The market is pricing K3 as a defeat for that architecture. SemiAnalysis's read: K3 is exactly the workload that justifies it.

Linear attention also does not retire the rest of the memory stack. Under concurrency, KV caches still spill to CPU DDR5 and NVMe storage when HBM runs out, so DRAM and SSD demand move with the model, not against it. The chip trade has been treating K3 as a one-direction short. The mechanism says the direction is sideways, with the mix shifting toward the most expensive components.

The thesis is falsifiable, and that is what makes it worth sizing. If WideEP turns out to be a K3-specific layout that other major labs do not adopt, the bears win and the rack-scale networking trade fades. If the next Chinese frontier release, or the next U.S. one, ships with a similar expert-parallel footprint, the SemiAnalysis read is the right one and the memory-chip selloff is the trade to fade.

The next data point is Moonshot's own disclosure of K3's serving footprint, expected in the next earnings cycle from NVIDIA and the memory vendors. Watch the rack-scale share of the order book. That number, not the next K3 demo, settles the bet.