High-bandwidth memory (HBM) is a specialised form of DRAM built by stacking memory chips vertically and connecting them with dense through-silicon wiring, then placing the stack directly beside a processor. The design moves data to and from AI accelerators far faster than conventional memory. Every serious AI chip, from NVIDIA’s GPUs to custom hyperscaler silicon, depends on it, and only three companies in the world can make it at scale: SK hynix, Samsung and Micron.
Why AI made HBM scarce
Training and running large AI models is constrained less by raw compute than by how quickly data reaches the processor, the so-called memory wall. Each generation of AI accelerator therefore carries more HBM per chip. HBM is also far harder to manufacture than standard DRAM: stacking and bonding yields are lower, production lines take years to qualify, and capacity converted to HBM cannot easily serve the ordinary memory market. Demand rising faster than the three suppliers can add capacity has made HBM the binding constraint of the AI build-out.
Why qualification matters
An accelerator maker cannot simply switch memory suppliers. Each HBM product must be qualified: tested against the specific processor, package and thermal envelope, a process that takes quarters. This gives incumbent suppliers pricing power once designed in, and it is why long-term supply agreements between chipmakers and memory producers effectively reserve future capacity years ahead.
What to watch
HBM supply agreements and prepayments announced by NVIDIA, AMD and the hyperscalers; SK hynix, Samsung and Micron capital expenditure guidance; DRAM contract pricing, which rises when capacity shifts to HBM; and qualification announcements for each new HBM generation. When memory makers say next year’s output is sold out, that statement sets the ceiling on how many accelerators can ship.