Beyond the GPU: storage is the AI line item most budgets still miss

The line item most AI cost models still underweight isn't the model or even the GPU. It's the per-request back-end storage that every inference call leaves behind. Operators who start modeling it now, before the storage line item becomes a rounding error turned structural, will be the ones who can price AI products honestly in 2027.

"Inference" is the work a model does each time it answers a user request, from chat replies to generated images and short video clips. Training builds the model once. Inference is the recurring bill. The industry has spent three years optimizing training cost per token, then per GPU-hour. It's spent almost no time optimizing the storage that every inference call writes, caches, and retains behind the user.

A short AI-generated video returned to a user as a 3.97MB 720p file can leave more than 500MB of associated data on the back end: logs, cached context, intermediate renders, backups, safety records, and sometimes redundant copies for compliance. That ratio, reported from the storage vendor's own internal measurements in a sponsored post, is roughly 130x the output file size for an 8-second clip. It's a single vendor's measurement of one workload, not a benchmark, and longer or higher-resolution clips scale the back-end footprint into the hundreds of gigabytes per generation.

The broader survey picture is more sober and comes from IDC's September 2026 white paper, "Built for Scale: The Enduring Role of HDDs in the AI Era," which a storage vendor sponsored. Of 763 IT and business decision-makers surveyed across seven countries, 94.7% reported that data volume grew in the past 12 months because of AI or generative AI, and 60.9% reported growth of 25% or more. Looking three years out, the white paper's executive summary puts the share expecting 25%-plus growth at 73.8%, with the share expecting 50%-plus growth nearly doubling from 16.1% to 30.5%. The same study finds 85.4% reporting that data lake volumes have grown over the past year, and 85.8% expecting further growth over the next three to five.

These are decision-makers reporting their own data, not vendor forecasts. They're also reporting under the sponsorship of a company that sells the storage their organizations will need. The numbers are real. The framing is a sales motion. Readers should treat both as fact.

The mechanism is what makes the storage line structural. GPUs are the specialized chips that do the math for AI, and they're expensive enough that an idle GPU is a line-item disaster. A senior technology executive at a hyperscale cloud service provider — identified in the study only by role, not by name — put it this way: "You can think of having more storage as an optimization for keeping the GPUs busy. The biggest challenge is to ensure that the GPUs are not idle, waiting for data to show up. Hours of GPU idle time becomes very expensive, very fast." That's also the vendor's pitch. It's also, on its own terms, true. A model that idles while data loads costs about as much as one idling for lack of a GPU. If storage spend grows 25% year over year on a path that compute spend grows on a predictable cadence, the rounding-error storage line becomes a structural fraction of program cost in 12 to 18 months.

McKinsey, as cited in the same sponsored post, projects AI inference demand to grow at a 35% compound annual growth rate through 2030, versus 22% for training, and to make up more than 40% of total data-center demand by the end of the decade. The underlying McKinsey insight page is not directly accessible to verification, so the projection should be read as McKinsey's, not as an industry consensus.

What an operator can actually do, starting now, is measure the line. The planning questions are specific. First, measure per-request back-end bytes, not output bytes, and track the ratio for each model and each modality, since a 130x ratio on a video model says little about a chat model's storage footprint. Second, audit log retention windows, which usually default to keep-everything; trimming them where compliance allows is the cheapest dollar saved. Third, watch cache hit rates on the inference path: a warm prompt cache is the cheapest storage an operator will ever buy, and a cold one is the most expensive. Fourth, treat archive retrieval latency as a planning constraint, especially for retrieval-augmented generation, where the model pulls stored documents to ground each answer. The same IDC study finds 96% of organizations expecting to need faster archive retrieval to support inference or RAG, and 75.9% pulling growing volumes of cold-tier data back into use for AI workloads. A cold tier that is now warm is a planning mistake waiting to happen. Fifth, track total cost of ownership per terabyte, weighted by tier; 98.2% of the surveyed decision-makers rated TCO per terabyte as important or very important, but few of them have a model for how that TCO moves when inference share grows.

The 130x back-end-to-output ratio is a vendor's number. The 94.7% AI-driven data growth and the 35% inference CAGR are credible signals from credible houses, with the funding relationships disclosed. None of them is the thesis. The thesis is mechanical: training was a one-time capex, inference is a recurring opex, and its largest under-modeled line is the storage that every call leaves behind. The builders who can price that line in 2026 will be the ones whose AI products are still profitable in 2027.