In plain words
A new model starts out knowing nothing. Its internal numbers, its parameters, are random, so every guess is a wild one.
Training fixes that through practice. Show the model an example. Let it guess. Compare the guess with the right answer and measure the gap. Then adjust every number slightly so that next time the gap is smaller.
Do this over thousands, millions or trillions of examples and the guesses get steadily better. The model is not memorising each example; it is settling on numbers that work across all of them.
Training a large model can take weeks on thousands of chips and costs a lot, so it is done once per version. Using the finished model afterwards is a separate, cheaper step called inference.
Everyday example
Bowling at a single stump
A kid bowls at one stump in the gully. Too far left, so next ball a little right. Too short, so next ball a bit fuller. Each miss tells her which way to adjust and roughly by how much. After hundreds of balls she hits the stump most of the time. Training is this loop done with numbers: try, measure the miss, correct, repeat.
Where the example breaks: The bowler adjusts one action and can feel why she missed. A model adjusts millions or billions of numbers at once, guided only by a formula for the miss, with no feel for what any single number means.
How it actually works
Most modern training is a loop built on gradient descent, run over a large dataset.
Forward pass: make a guess
A batch of examples from the training data goes through the model, which makes predictions using its current parameters.
Loss: measure the miss
A loss function scores how far each prediction is from the right answer. One number sums up how badly the model is doing.
Backward pass: share out the blame
Backpropagation works out, for every parameter, whether nudging it up or down would lower the loss, and by how much.
Update: nudge the numbers
Gradient descent moves every parameter a small step in the helpful direction. The size of that step is the learning rate.
Repeat until it stops improving
One full pass through the data is an epoch. Training stops when the loss levels off on data the model has not seen. Pushing further can cause overfitting.
- An early example
- 1959: Arthur Samuel's checkers program learned to beat its author after 8 to 10 hours of play
- Scale today
- Meta trained Llama 3.1 405B on over 15 trillion tokens using over 16,000 GPUs (July 2024)
- Stages for chatbots
- Pre-training first, then fine-tuning and RLHF
- Who can afford it
- Large models: mostly big labs, because of the computing cost
Questions people ask
What is the difference between training and inference?
Training is learning: adjusting the model's numbers using lots of examples. Inference is using: running the finished model on a new input to get an answer. Training happens once per version and is expensive. Inference happens every time someone uses the model.
How long does it take to train an AI model?
From seconds to months. A small model can train on a laptop in minutes. The largest language models train for weeks or months on thousands of specialised chips in data centres.
Does ChatGPT learn from my conversations?
Not in real time. The model's numbers are fixed while you chat. Depending on your settings and the company's policy, conversations may be used to train future versions. Most chat apps let you turn this off in their data controls.
What is pre-training vs fine-tuning?
Pre-training is the first, huge training run that gives a model broad knowledge, for example by predicting the next word across trillions of words. Fine-tuning is a shorter, later run on focused examples that shapes the model for a specific job or style.