In plain words
When people in AI say compute, they mean the amount of calculation a job needs, or the machines that supply it. Think of it like electricity: you can count it, buy it, and run out of it.
Building an AI model takes compute twice. First in training, when the model reads huge amounts of data and adjusts itself over weeks on thousands of chips. Then every time someone uses it, called inference, each answer needs its own burst of calculation.
Over the past decade, the surest way to make AI better has been to spend more compute on it. The compute used to train the top models has grown roughly fivefold every year since 2020. That is why AI companies race to buy chips and build data centers, and why compute has become a question of money, energy and even national policy.
Everyday example
Cooking for a wedding
Making dinner for four takes one cook and one stove for an hour. Feeding 2,000 wedding guests needs dozens of cooks and burners running all day. The recipe is not harder; there is simply far more cooking. Compute is the total cooking: number of burners times how fast each one works times how long they run.
Where the example breaks: A bigger kitchen only feeds more people. In AI, more compute can change what the model is able to do, not just how much it produces. And unlike cooks, chips need a smart recipe (good data and design) or the extra compute is wasted.
How it actually works
Compute is usually measured in FLOPs: the total number of arithmetic operations a job uses.
Three ingredients
Total compute is roughly the number of chips, times how many operations each does per second, times how many seconds they run. Faster chips, more chips or longer runs all add compute.
Training compute
Training a large model can take on the order of 10²⁵ operations or more. The scaling laws describe how much better a model gets as training compute grows, which is why labs plan in compute budgets.
Inference compute
Once a model is in use, every token it writes costs a fixed slice of calculation. Millions of users add up to a huge ongoing bill. Newer reasoning models also spend extra compute at answer time, thinking longer before replying.
Why it keeps growing
Labs are spending more each year, and each chip generation does more per rupee or dollar. Epoch AI estimates training compute for frontier models has grown about 5 times a year since 2020, while the cost of those runs grew about 3.5 times a year.
- 2012
- AlexNet, a landmark image model, trained for five to six days on two gaming GPUs
- Growth since 2020
- About 5x a year for frontier training runs (Epoch AI, as of February 2026)
- Biggest known AI site
- About 1.1 million H100-equivalent GPUs of compute (Epoch AI estimate, February 2026)
- A famous essay
- Rich Sutton's "The Bitter Lesson" (2019): methods that use more computation tend to win in the long run
Compute is now policy, too. The EU AI Act uses a training compute threshold (10²⁵ operations) to decide which models face extra rules.
Questions people ask
What does compute mean in AI?
Why does AI need so much compute?
Modern models have billions of parameters, and training adjusts all of them after reading each batch of data, trillions of words in total. Every adjustment is a pile of multiplications. Answering users then needs more calculation for every word produced.
Is more compute always better?
Usually, but not on its own. More compute helps most when it comes with enough good data and a well-designed model. The 2022 Chinchilla study showed many models were too big for their data, and a smaller model trained on more data did better for the same compute.
What is the difference between training compute and inference compute?
Training compute is spent once, to build the model. Inference compute is spent every time someone uses it. For a popular model, the total spent on inference over its lifetime can grow very large, because it is paid again for every question.