In plain words
A model learns from examples, and those examples always contain some noise: odd cases, typos, plain luck. A good model learns the general pattern and shrugs off the noise. An overfit model learns the noise too.
On the data it trained on, an overfit model looks brilliant, sometimes perfect. But real life sends it new examples with different noise, and its memorised quirks lead it astray. It has learned the answers, not the lesson.
That is why models are always checked on a test set they never saw during training. A big gap between training score and test score is the classic sign of overfitting. The opposite problem, a model too simple to catch the pattern at all, is called underfitting.
Everyday example
Mugging up last year's question paper
Two students prepare for an exam. One memorises last year's paper word for word, answers and all. The other works through the chapters until the ideas make sense. Give them last year's paper and both score full marks. Give them this year's paper, with new questions, and only the second one does well. The first one overfit.
Where the example breaks: A student can notice they are memorising and change approach. A model has no idea it is overfitting; people have to spot it by testing on fresh data and then change the model or the training.
How it actually works
Google's glossary defines overfitting as a model that matches the training data so closely that it fails to make correct predictions on new data. The goal is generalisation.
Why it happens
The model is too flexible for the amount of data: too many parameters, too few examples, or training for too many epochs. With enough freedom it can bend to fit every point, noise included.
How to spot it
Track loss on the training data and on a separate validation set. If training loss keeps falling while validation loss starts rising, the model has begun memorising.
More and better data
A larger, more varied training set makes memorising harder and the true pattern easier to see. Data augmentation creates extra variety, for example by flipping or cropping images.
Regularisation
Any technique that stops a model fitting too exactly. Examples: penalising large weights, dropout (randomly switching off parts of a neural network during training) and early stopping, which halts training when validation loss starts to rise.
Underfitting, the other side
A model that is too simple, trained too briefly or given poor features misses the real pattern and does badly everywhere. Good training sits between the two.
- Opposite
- Underfitting: too simple to capture the pattern
- Telltale sign
- Great training score, much worse test score
- Common fixes
- More data, data augmentation, regularisation, dropout, early stopping, a simpler model
- Goal
- Generalisation: correct predictions on new, unseen data
Questions people ask
What is overfitting in simple words?
When a model learns its training examples too exactly, including their random quirks, so it does well on those examples and badly on new ones. It is memorising instead of understanding the pattern.
What is the difference between overfitting and underfitting?
An overfit model is too tuned to its training data and fails on new data. An underfit model is too simple and does badly even on its training data. One learned too much detail, the other not enough.
How do you prevent overfitting?
Use more and more varied data, keep a separate test set to check, stop training when validation results stop improving, and use regularisation methods such as dropout. Sometimes a simpler model is the answer.
Can large language models overfit?
Yes. They can memorise and repeat passages that appeared many times in their training data, and a model tested on questions it saw during training can look far better than it is. That second problem is why benchmark contamination is watched closely.