The Power Grid Will Break Before the Transistors Do
Rambus, a Silicon Valley intellectual property company most people have never heard of, announced March 4 that it has built the controller silicon AI accelerators need to use the next generation of High Bandwidth Memory 4E, or HBM4E. The numbers are real. The physics are brutal. And the power grid is going to be the bottleneck before the transistors max out.
The HBM4E Memory Controller IP delivers 16 gigabits per second per pin, 60 percent faster than the HBM4 generation it sits above, Rambus said in its announcement. One HBM4E stack through this controller reaches 4.1 terabytes per second of memory bandwidth. Stack eight of them and an AI accelerator gets over 32 terabytes per second. Nvidia's current H100 delivers roughly 3.5 terabytes per second, SemiAnalysis noted. The next generation is an order of magnitude higher.
"HBM bandwidth is one of the main bottlenecks on LLM performance," Reiner Pope, co-founder and CEO at MatX, a startup building inference-optimized AI chips, said in Rambus's announcement. Pope's company competes with Nvidia. The bandwidth constraint is not a talking point.
The controller is the logic that sits between an AI chip and the stacked DRAM beside it on the same package, telling the memory when to send what and at what speed. It is not the memory itself. Samsung and SK Hynix both have HBM4E devices running at 11.7 gigabits per pin, according to EE Times, with Micron at 11 gigabits per pin. Rambus makes the bridge, not the memory.
At 16 gigabits per second, the controller design is no longer the hard part. The hard part is getting a clean signal across the interconnect between the PHY and the memory device: capacitance, parasitics, routing distance, and signal time-of-flight. Simon Blake-Wilson, senior vice president and general manager of Silicon IP at Rambus, did not sugarcoat it. "The explosion of AI workloads is driving unprecedented demand for memory bandwidth and capacity," he said in the announcement.
Rambus says the controller supports the 2,048-bit wide interface standard HBM4 established. The company has over 100 HBM design wins across previous generations, a track record that matters in chip IP because IP that works the first time is worth more than IP that is theoretically faster, per Rambus. The target ASICs, per EE Times, are expected in 2027-2028. HBM4E doubles the bandwidth of HBM4 while preserving its power efficiency and latency, SemiEngineering noted. At 16 gigabits per second per pin over a 2,048-bit interface, HBM4E delivers 4.1 terabytes per second per stack versus 2.56 terabytes per second for HBM4 at 10 gigabits per second. That 60 percent jump is the engineering headline.
The engineering consequence is the power bill. Nvidia's Rubin Ultra, arriving in 2027, is expected to use HBM4E, per Rambus. AMD's MI500 series is also targeted for HBM4E integration, per SemiEngineering and AMD's CES 2026 press releases. Two chips, two different memory specs, one controller that handles both.
Stacked DRAM cannot reliably operate above roughly 95 degrees Celsius, EE Times reported. Unlike processors, memory cannot be cooled the same way. At 32 terabytes per second across eight stacks, the cooling challenge scales with the bandwidth. Per-rack heat density for AI training clusters has already surged from the traditional 10-14 kilowatts to over 100 kilowatts, Tech Insider reported. That was for the current generation.
Most data center facilities cannot support racks over 30 kilowatts, Brightlio reported. Virginia, home to the world's largest data center market, has already hit a wall. Grid operators there issued formal capacity warnings through 2028 and several counties in Northern Virginia have halted new data center permits until power infrastructure catches up, Tech Insider reported. The memory bandwidth race is running into the power grid before the physics give out.
Rambus is shipping controller IP now. The memory vendors have devices that work. The ASICs that need to integrate both are still being designed. IP is ready. Silicon is not. Rambus has done this before. Whether the AI accelerator roadmap holds to the same schedule is a different question.
† Add attribution: 'Nvidia's current H100 delivers roughly 3.5 terabytes per second, according to [source],' or add '†Source-reported; not independently verified.'
†† Add citation to the specific AMD press release or SemiEngineering piece confirming MI500 HBM4E integration, or add '†Source-reported; not independently verified.'
††† Add attribution to a source confirming these product specs, or add '†Source-reported; not independently verified.'
†††† Add citation to BusinessWire, TechInsider, or another registered source, or add '†Source-reported; not independently verified.'
††††† Add citation to the specific source confirming Virginia grid warnings, or add '†Source-reported; not independently verified.'