The next 'TB GPU' is a tier question, not a capacity one

AI's memory problem has always been a tier problem, not a capacity problem. The Register's reporting on high-bandwidth flash surfaces the actual move: a third memory tier between HBM and SSD that absorbs the spillover a model triggers when it will not fit on-package. The naive read treats HBF as a HBM replacement, another path to a four-terabyte GPU. The structural read treats it as an admission that single-package capacity is structurally capped, and that inference economics live or die on the tier that catches the overflow.

The Register's caveat does the load-bearing work. High-bandwidth flash promises SSD-like capacities with HBM-like speeds, but it is not all unicorns and rainbows. That one line pins the two falsifiers any HBF announcement has to clear: NAND's worst-case latency under AI-inference duty cycles, and write endurance at the rates large-context serving demands. The IEEE H³ paper and the SanDisk and SK hynix standardization press release turn the hybrid stack from a single-vendor pitch into a coordinated academic-and-industry proposal.

HBM4 and HBM5 scaling remain the standing counterargument: if on-package capacity catches up fast enough, the third tier never has to ship. The next 'X-TB GPU' announcement will live or die on which tier actually owns the spillover.