Products, laws and numbers here change fast. Checked October 2026.
In plain words
Most AI runs on GPUs, chips first made for video games. In 2013, Google worked out that if people used voice search for three minutes a day, it would need to double its data centers. So it built its own chip, made only for AI maths.
That chip is the TPU, or tensor processing unit. A tensor is just a grid of numbers. A TPU is a big block of tiny calculators wired side by side. Numbers flow through it in waves, and each calculator multiplies, adds, and passes its result to the next one.
Because it gives up everything else a normal chip can do, a TPU can pack in more maths per watt. Google uses TPUs to train and run its own models, such as Gemini, and rents them to other companies through its cloud.
Everyday example
A bucket line at a well
A village needs to fill a tank. Instead of each person walking to the well and back, everyone stands in a line and passes buckets hand to hand. Nobody walks; each person does one small move, over and over, all at once. Water reaches the tank in a steady rhythm. A TPU passes numbers between neighbouring calculators the same way, so nothing travels far.
Where the example breaks: In a bucket line, each bucket is carried unchanged. In a TPU, every calculator also changes what it passes on: it multiplies two numbers and adds the result to a running total.
How it actually works
A TPU is an ASIC (application-specific integrated circuit): a chip built for one job. Its core is a matrix multiply unit arranged as a systolic array.
Neural networks are mostly matrix multiplication
Each layer of a neural network multiplies a grid of inputs by a grid of weights. That one operation dominates both training and inference. A TPU spends most of its silicon on it.
The systolic array
The first TPU had a 256 by 256 grid of 65,536 multiply-add cells. Weights sit in the cells; input numbers enter from one edge and step one cell per clock tick. Each cell multiplies, adds to the partial sum passing through, and hands both on. The name comes from the heartbeat (systole), because data pulses through in waves.
Read once, use many times
Fetching numbers from memory costs far more energy than multiplying them. In a systolic array each number is read once, then reused by a whole row of cells over short wires. That is where the efficiency comes from.
Low precision on purpose
The first TPU used 8-bit whole numbers instead of 32-bit decimals, since neural networks tolerate rounding. Later TPUs added other compact formats, such as FP8. Smaller numbers mean more cells fit on a chip. See quantization.
Pods: thousands of chips as one
Later TPUs connect directly to each other in pods. A seventh-generation Ironwood pod links 9,216 chips with fast chip-to-chip links, so one model can be split across all of them.
- In Google's data centers since
- 2015; announced publicly in May 2016
- Early uses
- Search ranking (RankBrain), Street View, and AlphaGo's 2016 match against Lee Sedol
- First TPU, measured
- About 15 to 30 times faster than the CPUs and GPUs Google compared it with on its own AI workloads (2017 paper)
- Seventh generation
- Ironwood (2025): 192 GB of fast memory per chip, pods of 9,216 chips
- As of October 2026
- Google announced the eighth generation in April 2026, split into TPU 8t for training and TPU 8i for inference
Questions people ask
What is the difference between a TPU and a GPU?
A GPU is a flexible parallel chip that started in graphics and now runs many kinds of work, including AI. A TPU is narrower: built only for the matrix maths of neural networks. That focus can give more work per watt, but TPUs are mostly available only through Google.
Can you buy a TPU?
Not the big data center ones. Google runs them in its own data centers and rents them by the hour through Google Cloud. Its Pixel phones do carry a Tensor chip with a small TPU inside, used for on-device AI features.
What does tensor mean in TPU?
A tensor is a grid of numbers with any number of dimensions: a list, a table, or a stack of tables. Neural networks pass tensors from layer to layer. Google's AI software, TensorFlow, and its chip both take their names from it.
Is Gemini trained on TPUs?
Yes. Google said in December 2023 that it trained Gemini 1.0 on its own TPUs, and it serves Gemini on TPUs too. Some other AI companies also rent TPUs from Google Cloud for their own models.
Further reading
- In-Datacenter Performance Analysis of a Tensor Processing Unit (Jouppi et al.)arXiv, 2017
- An in-depth look at Google's first Tensor Processing UnitGoogle Cloud, 2017
- Google supercharges machine learning tasks with TPU custom chipGoogle Cloud, 2016
- Introducing Gemini: our largest and most capable AI modelGoogle, 2023
- Google Cloud unveils eighth-generation TPUsTechRadar, 2026