In plain words
One decision tree is easy to read but easily fooled. It can latch onto quirks in its examples, and a small change in the data can give a completely different tree.
A random forest fixes this with a crowd. It grows many trees and makes each one a little different on purpose: every tree learns from its own random sample of the examples, and at each question it may only look at a random few of the clues.
To answer, every tree gives its own prediction and the forest combines them: the most common vote for a category, or the average for a number. Each tree makes its own odd mistakes, and those mistakes tend to cancel out, so the crowd is steadier and usually more accurate than any single tree.
Everyday example
Asking several shopkeepers
You want to know a fair price for a second-hand scooter. One mechanic might be off because of the few scooters he happened to see last month. So you ask several, each with different experience, and go with what most of them say. Individual quirks wash out. A random forest does this with trees, deliberately giving each one different examples so their errors differ.
Where the example breaks: Your mechanics have real, independent experience. The trees in a forest all learn from the same dataset, just shuffled samples of it, so if the data itself is biased or missing something, every tree shares that blind spot.
How it actually works
A random forest is an ensemble: many models combined into one. Google's course describes it as decision trees "trained with a specific random noise".
Bootstrap the examples
Each tree trains on a sample drawn from the training set with replacement, so some examples appear twice and others not at all. This is called bagging (bootstrap aggregating).
Randomise the questions
At each split, a tree considers only a random subset of the features. This stops a few very strong clues from making every tree identical, so the trees disagree in useful ways.
Grow deep, then combine
Individual trees are usually grown deep without pruning, so each one overfits a little on its own. Combining them, by majority vote for classification or averaging for regression, smooths those errors away.
More trees, no harm
Google's course notes that adding trees almost always lowers the error and cannot make the forest overfit; at some point it simply stops improving. Libraries such as scikit-learn average the trees' probabilities rather than counting single votes.
Trade-off
You lose the single readable flowchart of one tree. Forests are bigger and slower to run, and harder to explain.
- Introduced
- Leo Breiman, "Random Forests", Machine Learning journal, 2001
- Earlier idea
- Tin Kam Ho created random decision forests in 1995
- Two kinds of randomness
- A random sample of examples per tree, a random subset of features per split
- Good at
- Tables of data: customers, transactions, sensor readings, medical records
Questions people ask
What is a random forest in simple terms?
Many decision trees, each trained on a different random sample of the data, that each give an answer. The forest goes with the majority vote, or the average for numbers. The crowd is steadier than any single tree.
Why is it called a random forest?
Forest because it is made of many trees. Random because each tree sees a random sample of the examples and, at every question, only a random few of the features. That randomness keeps the trees different from one another.
What is the difference between random forest and gradient boosting?
A random forest grows its trees independently and averages them. Gradient boosting grows trees one after another, each fixing the mistakes of the ones before. Boosting can overfit as trees are added, so it needs more care; a forest does not.
Can a random forest overfit?
Adding more trees does not cause overfitting, but a forest can still learn the noise in a small or unrepresentative dataset. Always check it on a test set it has not seen.