AI Vocabulary · 02 AI Infrastructure
What is HBM (High-Bandwidth Memory)?
Very fast memory stacked right next to an AI chip, so its processors never wait for data. In 2025, it was a key bottleneck.
An AI chip can only calculate as fast as data reaches it. High-bandwidth memory, HBM, solves this by stacking memory chips on top of each other and placing them in the same package as the processor. A special packaging step connects the two, at TSMC under the name CoWoS.
HBM is made by a handful of memory companies, such as SK hynix, Samsung and Micron, and it is in short supply. Epoch AI estimated that in 2025 the four biggest AI chip designers used about 90 percent of the world's HBM and advanced packaging capacity, but only about 12 percent of advanced logic chips. Its conclusion: in 2025, memory and packaging limited AI chip production, not chipmaking itself.
Why it matters for you
Bottlenecks like HBM decide how many AI chips exist, and so how much AI capacity the world has. They are one reason prices and availability of AI services can shift quickly.
Own it
For running models yourself, memory is often the real limit, too: a model's parameters have to fit into the memory of your chip. Check memory first when you size hardware for self-hosting.
Where it comes from
HBM is an industry standard for stacked memory. The 2025 figures come from Venkat Somala, Advanced packaging and HBM, not logic dies, were the bottlenecks on AI chip production in 2025 (Epoch AI, 12 March 2026). They are estimates, measured by value.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.