In plain words
Picture a very simple weather forecast. If today is sunny, tomorrow is sunny 8 times in 10. If today is rainy, tomorrow is a coin toss. To guess tomorrow, you only need to know today. Last week doesn't matter.
That is a Markov chain: a set of states (sunny, rainy) and the odds of moving between them. Start anywhere, roll the dice at each step, and you get a believable sequence: sunny, sunny, rainy, rainy, sunny.
This one simple rule, "the future depends only on the present", turns out to be very useful. It has been used to model text, board games, genes, physics and the web. Google's original way of ranking web pages imagined a surfer clicking links at random, which is a Markov chain. Early text generators worked this way too, long before next-token prediction in today's chatbots.
Everyday example
Snakes and ladders
In snakes and ladders, where you go next depends only on your square and the dice. It doesn't matter whether you climbed a ladder or slid down a snake to get there. Each square is a state, each dice roll gives the odds of moving to other squares, and the whole board is a Markov chain.
Where the example breaks: Snakes and ladders really does forget the past. Real weather, language and people usually don't, so a Markov chain is a simplification that works only when the present state captures enough of what matters.
How it actually works
A Markov chain has a list of states and a table of transition probabilities: for each state, the chance of moving to each other state at the next step.
The Markov property
The defining rule: given the present state, the past adds no extra information about the future. This "memorylessness" is what keeps the maths simple enough to calculate with.
Transitions as a table
Each row of the table lists where you might go next from one state, and its numbers add up to 1. Multiply the table by itself and you get the odds two steps ahead, then three, and so on.
The long run
Run a chain for a long time and it often settles into a steady pattern, called a stationary distribution: the share of time spent in each state. Google's PageRank used this idea, treating the share of time a random surfer spends on each page as a measure of importance.
Making text
In 1948 Claude Shannon generated fake English by picking each next word using probabilities based on the word before. The result reads oddly but almost sensibly. Modern large language models also predict text one piece at a time, but they condition on thousands of earlier tokens through a transformer, not on one or two.
Relatives in AI
Hidden Markov models, where the true state is not seen directly, became a workhorse of speech recognition, set out in a 1989 tutorial by Lawrence Rabiner. Markov chains also power Monte Carlo methods that draw representative samples from huge sets of possibilities.
- Introduced by
- Russian mathematician Andrey Markov, starting in 1906
- First famous use
- In January 1913 Markov analysed the first 20,000 letters of Pushkin's poem Eugene Onegin, counting how often vowels and consonants follow each other
- Text generation
- Claude Shannon's 1948 paper "A Mathematical Theory of Communication" built English-like text from Markov processes
- On the web
- Google's PageRank modelled a "random surfer" clicking links, a Markov chain over web pages
Questions people ask
What is a Markov chain in simple words?
A model of something that moves from state to state, like weather from sunny to rainy, where the next step depends only on the current state. It ignores how you got there.
Is ChatGPT a Markov chain?
Not in the usual sense. A simple Markov text generator picks the next word from the last word or two. ChatGPT predicts each next token using the whole conversation so far, thousands of tokens, with a transformer. Both write one piece at a time from probabilities.
What is the Markov property?
The rule that the future depends only on the present, not on the path that led there. Given today's state, knowing yesterday's adds nothing to the forecast.
Where are Markov chains used?
In modelling board games, physics, genetics and text, in older speech recognition systems, and in Google's original PageRank. They also power a family of methods for drawing representative samples from huge sets of possibilities.
What is a hidden Markov model?
A Markov chain where you can't see the states directly, only clues they produce. For example, in speech you hear sounds but not the words that produced them; the model infers the most likely hidden sequence.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change