AIBRIEF
Multiverse Computing says caching only a teacher's top 100 guesses per token cuts the GPU bill for knowledge distillation, training a small model to mimic a larger one.
Training a small AI to imitate a bigger one, called knowledge distillation, has long demanded GPUs as large as the teacher's. Multiverse Computing says it just cut that bill.
In a Hugging Face post and arXiv paper this week, the San Sebastián, Spain-based lab described two changes. The teacher's full per-token guess is replaced by a cached list of its top 100 guesses, so the teacher can leave training memory. A chunked loss never builds the full logit tensor, so peak memory now grows with sequence length instead of vocabulary size.
On a single H200, the authors report distillation runs roughly 29% faster per step and up to 41% higher throughput at the same training loss. Distilling a 20-billion-parameter model at 32,768-token context shrank from four GPU nodes to one, with step time falling from 57 seconds to about 12, roughly 5× faster.
That cost is now the bottleneck as open-weights frontier models like Kimi K3 at 2.8 trillion parameters are too large to deploy as-is. The release puts long-context distillation within reach of smaller labs and open-weights projects that cannot keep a frontier teacher loaded next to the student.
Two caveats. The student still trails the teacher, with Multiverse reporting a roughly 9-point MMLU gap at less than half the parameter count, and the chunked-loss code carries a pending patent, EP26382987.1. Cheaper, not free.