In plain words
In 2020, researchers trained many language models of different sizes and measured how often each one guessed the next word wrong. They found a surprisingly tidy pattern. Every time they multiplied the computing power by ten, the error dropped by about the same fraction.
That pattern is called a scaling law. It is not a law of nature, like gravity. It is an observed trend, but a very steady one, and it held across a huge range of sizes.
Its practical power is prediction. A lab can train a few cheap small models, draw the line, and estimate how good a giant model will be before spending months and a fortune on it. Scaling laws are a big reason AI companies kept building larger models and data centers: the line said it would pay off.
Everyday example
Practice and your typing speed
Someone learning to type tracks their mistakes. Ten hours of practice takes them from 20 errors a page to 10. A hundred hours takes them to 5. A thousand, to about 2.5. Each tenfold increase in practice roughly halves the errors. Once you see that pattern, you can predict where ten thousand hours would get you, without doing them first.
Where the example breaks: A typist eventually hits a floor and real life is messier. Scaling laws also flatten at some point and only predict the model's raw word-guessing error, not which specific skills will appear or whether the model will be useful.
How it actually works
Scaling laws are fitted curves linking a model's test loss (its average error at predicting the next token) to the resources used to train it.
A power law
Kaplan and colleagues at OpenAI (2020) found loss falls as a power law in three things: the number of parameters, the amount of training data, and the training compute. Plotted on log axes, each is close to a straight line.
Shape matters less than scale
The same paper found details such as how deep or wide a network is matter little within a wide range. Total size and compute dominated.
Chinchilla: balance size and data
In 2022, DeepMind's Chinchilla study found that for a fixed compute budget, model size and training tokens should grow together. Its 70 billion parameter model, trained on four times more data, beat the 280 billion parameter Gopher that used the same compute.
Using the laws to plan
Labs fit curves on small runs, then pick the best mix of model size and data for a big run's budget. This turns a gamble costing millions into a forecast.
A second axis: thinking time
In 2024, OpenAI reported that its o1 reasoning model also improved steadily with more compute spent at answer time, thinking longer before replying. Scaling now happens in training and in inference.
- Landmark paper
- "Scaling Laws for Neural Language Models", Kaplan et al., OpenAI, January 2020
- Key revision
- "Training Compute-Optimal Large Language Models" (Chinchilla), Hoffmann et al., DeepMind, March 2022
- What they predict
- Average test loss, a measure of prediction error, not specific abilities
- Trend in practice
- Compute for frontier training runs has grown about 5x a year since 2020 (Epoch AI, February 2026)
Whether the curves keep holding at ever larger scale, and whether lower loss keeps turning into more useful models, is an open and much argued question.
Questions people ask
What are scaling laws in simple terms?
They are an observed pattern: as you give an AI model more size, more data and more computing power, its error goes down by a predictable amount. That lets researchers forecast how good a big model will be from small experiments.
Are scaling laws real laws, like physics?
No. They are fitted to experiments, not derived from first principles. They have held across many orders of magnitude, which is why people trust them, but nothing guarantees they continue, and they can change with new methods or data.
What is the Chinchilla scaling law?
A 2022 DeepMind finding that, for a fixed compute budget, model size and amount of training data should be increased in equal proportion. Many earlier models were too large for the data they saw. Chinchilla, smaller but trained on more data, beat a model four times its size.
Is AI scaling hitting a wall?
People disagree. Gains from simply making models bigger have become costlier, and good training data is limited. At the same time, new axes such as more compute at answer time (reasoning models) keep giving steady improvements. The honest answer as of October 2026 is that nobody knows where the curves end.