Nvidia's Kyber delay exposes a bigger AI bottleneck: the wiring between chips
Kyber, Nvidia's rack scale AI system, just slipped to 2028, and the delay exposes where the AI bottleneck has moved: from faster chips to the wiring between them.
Kyber, Nvidia's rack scale AI system, just slipped to 2028, and the delay exposes where the AI bottleneck has moved: from faster chips to the wiring between them.
Nvidia's Kyber NVL144 rack system, the company's next-generation platform for stitching AI chips into one giant computer, has slipped by more than a year and now targets 2028, according to SemiAnalysis. The delay is more than a supply chain hiccup. It surfaces a constraint the AI industry has been arguing around for two years, according to analyst Big Boss (@0xBADB01E): the next round of AI cost wins will not come from faster chips or fatter memory. They will come from the wiring between them.
The Kyber system is built from Nvidia's Rubin chips, the successor to Blackwell, and is meant to act as a single AI supercomputer across 144 GPUs. SemiAnalysis also reported that Nvidia has cancelled its NVL72x2 back-to-back rack architecture, leaving Rubin Ultra with a smaller "scale-up" domain than originally planned. The combined effect: each generation of Nvidia hardware fits inside a tighter box, and the box is held together by a fabric that is starting to look like the bottleneck.
The fabric, called the interconnect, is the chip-to-chip wiring inside a server and across servers in a rack. On Blackwell, that interconnect, branded NVLink, runs at about 1.8 terabytes per second, while the memory sitting next to each chip moves data at roughly 8 terabytes per second, according to Big Boss. The result is a permanent traffic jam: every time a chip wants to read new weights from a neighbor's memory, it waits. The waiting is most visible during inference, which is what happens after a model is trained and is asked to answer questions one token at a time.
For each token, the chip has to pull the next batch of weights from its neighbors. Because the interconnect is roughly four times slower than local memory, Big Boss estimates autoregressive decode on Blackwell runs at under 20 percent of peak FLOPs, the rough measure of a chip's computational throughput. The other 80 percent of the chip sits idle, drawing power while it waits for data.
Nvidia originally designed NVLink to handle training, where thousands of GPUs exchange gradients in parallel and can tolerate some latency. Inference is a different shape: latency matters per token, and the chips spend more time waiting on data than computing on it. Big Boss argues that the cure is "disaggregated memory," a design in which a chip reads weights streamed directly from a remote memory pool over the interconnect, rather than waiting on local HBM (high-bandwidth memory, the fast memory stacks attached to AI chips). The proposed silicon target is an interconnect that runs as fast as SRAM, the on-chip cache inside processors (around 80 terabytes per second), as broad as HBM (a few terabytes per second), and as cheap as LPDDR, the lower-power DRAM used in phones and some accelerators (a few hundred gigabytes per second), an aggressive combination that no shipping product achieves today.
Independent commentator jwt0625 corroborates the framing, noting that chipmakers have under-invested in the chip-to-chip fabric relative to the rest of the silicon and that the constraint now extends from the package outward to rack-to-rack links. CNBC's re-reporting of the Kyber delay and CryptoBriefing's coverage anchor the news in mainstream tech and finance press. The structural argument still rests mostly on one analyst's thesis, and Nvidia has not publicly disputed the Kyber timing or the NVL72x2 cancellation as of writing.
If the bottleneck has moved to interconnect, the next round of competition plays out on a different axis than the last one. AMD's MI400 platform, Marvell's custom silicon business, and Broadcom's hyperscaler programs at Google and Meta are sold partly on whether they can ship more fabric per chip than Nvidia. The custom silicon built inside the hyperscalers themselves, including the Trainium and Inferentia lines at Amazon, the TPU program at Google, and the Maia effort at Microsoft, will be evaluated on whether their fabric lets an operator amortize one trained model across many cheap inference boxes, or whether the rack becomes the new bottleneck instead of the old one.
Two things to watch. First, Nvidia's GTC keynote later this year, where the company is expected to walk back the Kyber timeline and frame Rubin Ultra around a smaller scale-up domain. Second, any hyperscaler disclosure on rack-level inference costs, which would either confirm or break the under-20-percent utilization thesis.