The Bottleneck Moved From Silicon to Metal

AI scaling is now a wiring problem. The chip has stopped being the bottleneck; the metal between chips has become it, because the loss budget over copper at 200-gigabit-plus signaling no longer closes inside the rack. Every extra meter of copper eats a piece of signal no amount of equalization can give back, and the power to push data through that wire is consuming the efficiency the GPUs are meant to deliver.

The clean rule is breaking in real time. Scale-up, the short copper links tying processors together inside a rack, is being forced to give ground to optics, and the old three-tier hierarchy has to absorb a fourth layer: bandwidth that stays inside a single board, so new that vendors only coined a name for it three months ago. SemiEngineering's reporting on Rambus, Omnitron, and Ayar Labs frames this as a one-line category blur. The underlying move is larger. Once optics is forced inside the rack to chase the loss budget, the rest of the architecture has to follow.

The repeatable mechanism: at every step up in signaling rate, copper's reachable distance and power cost shrink; the workloads keep demanding more bandwidth inside tighter footprints; so optics has to follow the bandwidth inward. Each generation of optics gets cheaper, but it must keep winning on power and reliability at ever-shorter distances, or the naming exercise outruns the engineering. Optics wins by physics, not by vendors renaming a layer.

For now, Google is the only production-scale proof point, and the rest of the industry is arguing about which rack to wire next.