Samsung puts AI math inside cheap DRAM to break HBM's grip on inference

The memory chips that feed AI accelerators have become the most expensive part of the system, and the top tier, high-bandwidth memory (HBM), the stacked DRAM that pairs with GPUs in data centers, is the part that is running out. At Hot Chips 2026, Micron warned that demand for HBM wafer capacity is only getting worse, even as memory's share of total AI chip cost keeps climbing. Samsung's answer, also unveiled at the same conference, is to take the math off the GPU entirely and push it into the memory chip itself.

The product is LPDDR5X-PIM, the first processing-in-memory (PIM) device built on the low-power DRAM standard that ships in phones, laptops, and edge AI boxes. PIM places a small block of logic next to the memory cells so the multiplications and additions that dominate inference happen inside the chip, eliminating the round-trip to the processor. For AI workloads overall, Samsung says LPDDR5X-PIM runs 3.01x faster than plain LPDDR5X with 8x the bandwidth; in inference tasks specifically, the company claims 2.28x. All of those numbers are Samsung's own from the Hot Chips presentation, and independent third-party benchmarks are not yet available.

Samsung has been iterating on PIM since 2021, starting with research demos, then a 2023 HBM-PIM product that AMD accelerator stacks adopted, and now LPDDR5X-PIM, the first version built on cheap DRAM rather than HBM. The HBM version had to carve space out of each memory bank to fit the logic. LPDDR5X-PIM gives every bank its own PIM block instead. The bank itself, a scale register, and a source register file feed parallel multiply-accumulate trees, and the output (integer or floating point) writes to a vector register file the host can read like a normal DRAM response. Two operating modes, single-bank for ordinary access and multi-bank for PIM compute, let a conventional DRAM controller switch between them using standard read/write and PIM read/write commands.

LPDDR5X-PIM drops into existing low-power DRAM sockets without requiring a new controller or interface, which is what makes the cost story credible. HBM is too expensive and capacity-constrained to feed every inference workload, and the workloads that do not need a top-end GPU accelerator are the ones with the most room to grow. Recovering inference throughput without paying the HBM bill is the actual value proposition. The Hot Chips symposium is peer-reviewed, which is a credibility signal, but the design still has to work in real customer boards, not just Samsung's own validation, before the economics actually change.

PIM does not replace HBM in the workloads where HBM already wins. Training large models, the regime that justifies HBM's cost, still needs the tight coupling between memory and the GPU that HBM provides. LPDDR5X-PIM competes for the long tail of inference: recommendation systems, on-device assistants, edge deployments, and any workload where the model is too big to fit in cache but the cost-per-query has to be cheap. That is the second-order effect: the addressable memory tier below HBM gets a compute upgrade at the same time as it gets a cost-down story.

Reordering requests against a conventional DRAM controller does not natively accommodate parallel MAC trees firing across multiple banks at once. Memory bandwidth only matters if the system can still schedule work efficiently, and that scheduling is what HBM's tight coupling to the GPU hides. Until the standard DRAM interface and the PIM compute path agree on ordering, software will have to do the work, and that is the watch item for the next twelve months.

Samsung is pushing for an initial JEDEC specification for LPDDR6X-PIM this year, which would put processing-in-memory on a standards-body roadmap for the first time. Independent benchmarks on real inference workloads, not Samsung's selected traces, will follow. Real customer deployment at scale, beyond Samsung's own validation, will close the loop.