In plain words
Type "Mumbai is the city of" and most of us think "dreams". We have seen that phrase so often that the next word almost writes itself.
A large language model does this as its one core skill. It looks at all the text so far and gives every possible next piece a score: "dreams" very likely, "lights" less, "joy" a little. Then it picks one, adds it to the end, and asks the same question again with the longer text.
Every answer a chatbot gives, from an email to a poem to working code, is built this way, one token at a time. It looks like planning because predicting well often needs a sense of where the sentence is going.
The model learned this by reading huge amounts of text and practising the guess billions of times, checking each guess against the word that really came next.
Everyday example
A word-by-word story game
Imagine a game where friends build a story together, each adding one word. To choose a good word, you listen to everything said so far and pick what fits best. Then the next person does the same with the longer story. A language model plays this game alone, very fast: read everything so far, add the best-fitting word, repeat.
Where the example breaks: Friends in the game know what a story is and where they want it to go. The model has no goal of its own beyond the next piece. Any plan exists only as patterns that make some continuations more likely than others.
How it actually works
The idea is old: Claude Shannon studied how well people guess the next letter of English text in 1951. Modern models do it with a transformer and enormous amounts of text.
Scores for every token
The final layer of the model outputs one score for every token in its vocabulary, often 100,000 or more. A function called softmax turns these scores into probabilities that add up to 100%.
Pick one
The app then chooses a token. Always taking the top one makes dull, repetitive text, so most apps sample: likely tokens usually win, but not always. A setting called temperature controls how adventurous that choice is.
Feed it back in
The chosen token is added to the text and the whole model runs again. This loop is called autoregressive generation. It stops when the model predicts a special end token or hits a length limit.
How it is trained
During pre-training, the model reads real text and predicts each next token. Its guess is compared with the actual token, and its parameters are nudged to make the right one a little more likely. Every position in every document is a free practice question, so no human labelling is needed.
Shaped into an assistant
A raw model just continues text. Instruction tuning and RLHF change which continuations are likely, so that the most likely next words form a helpful answer.
- Earliest roots
- Claude Shannon, 1951: measured how well people can predict the next letter of English text
- GPT-2, 2019
- OpenAI trained it "simply to predict the next word in 40GB of Internet text"
- GPT-3, 2020
- 175 billion parameters, trained on 300 billion tokens with the same objective
- Also called
- Autoregressive language modelling, or causal language modelling
Because each token is chosen by probability, a fluent but false sentence can be just as likely as a true one. That is one root of hallucination.
Is predicting the next word enough for understanding?
One view
Just autocomplete
The model only learns which text tends to follow which. It has no contact with the world the words describe, so whatever it seems to understand is a statistical echo of human writing.
The other view
Prediction forces understanding
To predict text about maths, code or people really well, a model has to build useful internal models of those things. In one 2022 study, a small model trained only on Othello moves was found to track the board inside it.
Most likely: Most researchers agree the training goal is next-token prediction, and that models still learn surprisingly rich internal structure to meet it. Whether that counts as understanding depends on how we define the word.
Questions people ask
Is ChatGPT just fancy autocomplete?
Mechanically, it writes by next-token prediction, like phone autocomplete. But the model is vastly larger, reads thousands of words of context at once and has been tuned to follow instructions, so it can do things a keyboard suggestion never could. "Just" undersells it; "autocomplete" describes the mechanism.
Why does the same question get different answers?
The model gives probabilities, and the app usually samples from them instead of always picking the top token. Small differences early on lead to different sentences later. Lowering the temperature makes answers more repeatable.
How can predicting one word at a time solve hard problems?
Each predicted token can carry reasoning forward. When a model writes out steps, it can read its own earlier steps before predicting the next one. That is the idea behind chain of thought and reasoning models.
Does the model plan the whole answer before writing?
Not as a separate step. It produces one token at a time. But studies that look inside models find some planning: in 2025, Anthropic reported that Claude picks a rhyming word before writing the line of a poem that ends with it. There is no hidden finished draft, though.