In plain words
Training or running a big AI model takes many GPUs working as one team. Every fraction of a second they must swap their partial results. The wiring that carries those results is called the interconnect, and its speed decides how much time the chips spend working instead of waiting.
There are two layers. Inside a server or a rack, chips sit close together and talk over short, extremely fast links. Nvidia's version is called NVLink. Its newest racks join 72 GPUs so tightly that software can treat them almost like one giant chip.
Between racks the distances are longer, so data travels over networks such as InfiniBand or fast Ethernet. These carry less per chip but can link tens of thousands of chips across a data center. A big training cluster needs both layers.
Everyday example
A wedding kitchen
Picture the kitchen at a big wedding. Cooks at one long counter pass bowls hand to hand, so they work like one team. That is NVLink inside a rack. Food for the other halls goes out on trolleys down long corridors: slower, but it reaches every hall. That is the network between racks. If the trolleys jam, the cooks stand idle, however fast their hands are.
Where the example breaks: Cooks pass finished dishes. Chips pass millions of half-finished numbers, and usually all of them must arrive before anyone can take the next step, so one slow link holds up the whole team.
How it actually works
AI chips can calculate faster than they can be fed, so in big systems moving data often limits speed more than doing the maths.
Why chips must keep talking
When one model is split across many chips, they regularly combine their results in an operation called all-reduce. In training this happens after every batch of examples. When a very large model is split across chips for inference, they exchange results inside every layer. The slower the link, the longer every chip waits.
NVLink: scale-up inside the rack
NVLink is a direct chip-to-chip link. The first version, on Nvidia's 2016 P100 GPU, gave each GPU 160 GB per second. The fifth generation, on Blackwell GPUs, gives 1.8 TB per second, more than ten times the first version. Nvidia says its sixth generation has 12 times the bandwidth of PCIe Gen6, the standard slot that connects a GPU to the rest of a computer.
Switches make one big domain
NVLink Switch chips connect every GPU in a rack to every other GPU. In an NVL72 rack, 72 Blackwell GPUs share 130 TB per second in total. The switches can even add numbers together as they pass through, which saves the chips another round trip.
Scale-out: InfiniBand and Ethernet
Between racks, each GPU usually gets its own network card. InfiniBand, first specified in 2000, has long been used in supercomputers, and Nvidia agreed to buy its main supplier, Mellanox, in 2019. Fast Ethernet does the same job, and Meta has used both for its Llama 3 models. These networks reach far more chips but carry less per chip.
Open alternatives
Other chipmakers want a shared standard instead of one company's link. The UALink group, formed in 2024 by firms including AMD, Google, Intel, Meta and Microsoft, published version 1.0 in April 2025. It is designed to join up to 1,024 accelerators in one pod.
- First NVLink
- Nvidia Tesla P100 GPU, April 2016: 160 GB per second per GPU
- Blackwell (fifth generation)
- 1.8 TB per second per GPU; 72 GPUs per NVL72 rack, 130 TB per second in total
- As of November 2026
- Nvidia's NVLink page lists sixth-generation NVLink, for Vera Rubin, at 3 TB per second per GPU (preliminary); its January 2026 launch announcement said 3.6 TB per second. First Rubin samples reached customers in early 2026, with production shipments planned for the second half
- Open rival
- UALink 1.0 (April 2025): up to 1,024 accelerators, backed by AMD, Google, Intel, Meta, Microsoft and others
The same idea applies to every AI chip, not only Nvidia's: Google's TPU pods use their own chip-to-chip links.
Questions people ask
What is NVLink in simple words?
NVLink is a very fast private connection between Nvidia GPUs. Instead of sending data through the computer's general-purpose slots, GPUs pass it to each other directly, so a group of them can work on one AI model with less waiting.
Is NVLink the same as PCIe?
No. PCIe is the general slot that connects a GPU to the rest of the computer, such as the processor and storage. NVLink is a separate link built for GPU-to-GPU traffic. Nvidia says its sixth-generation NVLink has 12 times the bandwidth of PCIe Gen6.
What is the difference between NVLink and InfiniBand?
NVLink joins GPUs inside a server or rack, over short distances at very high speed. InfiniBand is a network that joins many racks across a data center. Big AI systems use both: NVLink to scale up, InfiniBand or Ethernet to scale out.
Why does AI need such fast connections?
Because a large model is split across many chips that must constantly share results. If the links are slow, expensive chips sit idle waiting for data, and training or answering takes longer and costs more.
Can other companies' chips use NVLink?
Mostly NVLink connects Nvidia's own GPUs and processors. Nvidia also offers NVLink Fusion, which lets large cloud companies pair NVLink with their own semi-custom chips. Rival firms are backing an open standard, UALink, instead.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change