Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

HBM

Also called high bandwidth memory, AI memory chip

HBM is a stack of memory chips placed right beside an AI chip, so the chip can be fed numbers fast enough to stay busy.

Chips and scaleTechnical

Products, laws and numbers here change fast. Checked October 2026.

In plain words

An AI chip such as a GPU can do trillions of sums a second. But first it needs the numbers: the model's parameters and your words. Those live in memory, and if memory is slow, the chip sits waiting.

HBM, short for high bandwidth memory, fixes this by changing the shape. Instead of memory chips spread around a circuit board, thin memory chips are stacked on top of each other like a pile of biscuits, and the stack sits right beside the AI chip. Thousands of tiny connections link them, so huge amounts of data can flow at once.

That is why a top AI chip today is ringed by several HBM stacks. Bandwidth, how much data moves each second, often decides how fast a chatbot can reply, as much as the chip's raw speed does.

Everyday example

The dosa counter

A dosa counter has a fast cook, but the batter is kept in a storeroom down the corridor, and one helper fetches one bowl at a time. The cook spends most of the evening waiting. Put a tall rack of batter tubs right beside the griddle, with many hands passing them across at once, and the cook never stops. HBM is that rack: lots of memory, stacked tall, right next to the chip.

Where the example breaks: Nearness is only part of it. HBM's real advantage is width: thousands of connections carrying data side by side, which only works because the stack sits so close to the chip on a shared base.

Go deeper

How it actually works

AI chips are often held back not by maths but by how fast they can read memory. Engineers call this the memory wall, and HBM is the main way around it today.

  1. Stacked chips

    An HBM stack is several DRAM memory chips, the same basic kind as in a laptop, thinned and piled up to 16 high. Tiny vertical wires called through-silicon vias run through the layers to connect them.

  2. Wide, not just fast

    Ordinary graphics memory talks over narrow 32-bit channels. Each HBM stack has a 1,024-bit interface, and the newest version, HBM4, doubles that to 2,048 bits. More lanes means far more data per second.

  3. Sitting on a shared base

    That many wires cannot run across a normal circuit board. The GPU and its HBM stacks sit side by side on a thin slice of silicon, called an interposer, that carries the connections between them.

  4. Why chatbots need it

    Each time a language model writes a new token, the chip must read through the model's weights again. That step is limited by memory bandwidth more than by maths, so faster HBM means faster replies.

  5. Few makers, high demand

    The main makers are SK hynix, Samsung and Micron. AI demand has outrun supply: in February 2026 Micron reportedly had already sold all the HBM4 it could make that year.

Became a standard
October 2013, adopted by the industry body JEDEC; developed by AMD with SK hynix
First product
AMD's Fiji graphics chip, June 2015
Newest standard
HBM4, April 2025: up to 2 TB per second and 64 GB per stack
Example chip
Nvidia's Blackwell B200 was announced in 2024 with 192 GB of HBM3e at 8 TB per second
As of October 2026
Samsung and Micron said in February 2026 they had begun volume production and first shipments of HBM4; Nvidia's Rubin GPU, which began reaching customers in 2026, carries 288 GB of HBM4

Phones and laptops use cheaper, slower memory; HBM is found mainly in data center AI chips.

Questions people ask

What is HBM used for?

Mainly for AI chips and supercomputers in data centers. Training and running large AI models means reading billions of numbers again and again, and HBM supplies them fast enough to keep the chip busy. Some high-end graphics cards have used it too.

What is the difference between HBM and normal RAM?

Both use the same basic kind of memory cell, DRAM. Normal RAM spreads chips across a board or a stick and talks over narrow channels. HBM stacks the chips into a tower beside the processor and links them with over a thousand connections per stack, so it moves far more data per second.

Is HBM the same as VRAM?

VRAM just means memory attached to a graphics chip. Most gaming graphics cards use GDDR memory chips laid out on the board around the GPU. Data center AI GPUs use HBM stacks instead, which are much wider and faster, and harder to make.

Who makes HBM?

As of October 2026 the main makers are SK hynix and Samsung, both of South Korea, and Micron of the United States. Chip designers such as Nvidia, AMD and Google buy the stacks and mount them next to their own AI chips.

Why is HBM part of export controls?

Because AI chips are useless without fast memory. In December 2024 the US added HBM to its controls on exports to China, alongside chipmaking tools. See chip export controls for how those rules work.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.