Kimi K3 Has Outgrown a B200 Node, and a Chip Buyer's Own Numbers Put AMD Ahead on Cost

A 2.8-trillion-parameter open-source AI model from Moonshot cannot live on a single node of the chip that runs most production inference today. The model, Kimi K3, needs more memory than 8 of NVIDIA's B200 GPUs can supply, and that physical limit is reshaping how frontier open-source models are procured.

That shift is now showing up in cost. Wafer, a chip buyer that runs its own AI infrastructure, published a benchmark putting AMD's MI355X ahead of NVIDIA's B300 on tokens per dollar when serving Kimi K3. The headline figure, 48 tokens per second per dollar of GPU-hour for the MI355X versus 33 for the B300, is Wafer's own measurement on its own deployment. It is also the first publicly attributed cost comparison of a frontier open-source model on the two latest accelerator generations, and the story it tells is about memory, not compute.

VRAM is the new constraint. A B200 ships with 192GB of memory per GPU; an 8-GPU node holds about 1.5TB. Kimi K3's 2.8 trillion parameters, at the 8-bit precision used for inference, weigh roughly 2.8TB. One full copy does not fit. NVIDIA's B300 raised per-GPU memory to 288GB, putting an 8-GPU node at about 2.3TB, enough to hold the model, and AMD's MI355X matched that 288GB per GPU, according to Moonshot's own model card on Hugging Face and Wafer's writeup.

Wafer's benchmark, run on 1,024-token inputs with 400-token outputs, measured three configurations. The 8x MI355X, running tensor parallelism 8 (TP8), hit 952 tokens per second aggregate and 118 tokens per second single-stream decode. The 2x8 B200 baseline, running TP16 split across two physical nodes, hit 498 tokens per second aggregate (about 249 per node) and 90 tokens per second single-stream decode. The B300, running TP8 plus DCP8, hit 1,568 tokens per second aggregate and 172 tokens per second single-stream decode.

The MI355X is the slowest of the three on raw throughput, but the cheapest per chip. Wafer puts MI355X GPUs at roughly 2.4x cheaper per unit than the B300 and 1.7x cheaper than the B200. On its peak-aggregate-per-dollar-per-GPU-hour metric, the MI355X reaches 48 tokens per second per dollar, against 33 for the B300. The performance-per-dollar lead is the headline.

That lead comes with caveats. The three configurations are not strictly comparable. The B200 baseline uses tensor parallelism 16 split across two physical nodes. The B300 uses TP8 plus a separate context-parallelism axis. The MI355X uses TP8. Different parallelism strategies stress different parts of the hardware, and the B200 number reflects inter-node communication cost the others do not pay. Wafer published the comparison anyway, with the TP regime stated on each row.

A separate technical concern surfaced on Hacker News. Wafer's MI355X kernel path pads Kimi K3's 12 attention heads up to 16, the head count the fast kernel expects. Commenters in the thread asked whether the zero-padded heads can produce coherent output at long context, or whether the workaround is a soft correctness tax. Wafer has not published a long-context quality evaluation against the head-count pad. Any buyer reading the 48 tokens-per-second figure should treat coherence on long inputs as an open question.

The shift the benchmark points to is real even with those caveats. AMD shipped day-0 inference support for Kimi K3, according to Wafer, removing a software-stack objection that historically made non-NVIDIA deployments a non-starter for frontier models. For most of 2024 and 2025, by Wafer's account, an open-source model of Kimi K3's size would have run on CUDA-only kernels, with ROCm support arriving weeks or months later. Day-0 support is the change, and it is what makes the cost comparison legible in the first place. Without it, the AMD price advantage would have been a paper number.

The remaining question is whether Wafer's measurement holds up under independent replication. No third party has published matching numbers on the same workload. The 48 tokens-per-second-per-dollar figure is one chip buyer's measurement of its own hardware, and it is the only such measurement in public.

Until that changes, the story is narrower than the headline suggests. Kimi K3 does not fit on a single B200 node. AMD's MI355X does, and a chip buyer running both reports it is cheaper per useful token.