In plain words
Some questions want a category as the answer. Others want a number: how much will this flat sell for, how many minutes until the food arrives, how much rain tomorrow. Predicting a number is called regression.
The simplest version draws a line through past examples. Plot used scooters by age and price, and the dots slope downwards. A line that runs through the middle of them sums up the pattern: each extra year knocks off roughly the same amount.
To price a new scooter, find its age on the line and read off the price. Real models use many inputs at once, such as age, kilometres, brand and city, and can bend instead of staying straight. But the job is the same: inputs in, a number out.
Everyday example
Guessing a fair price for a used scooter
Before selling her scooter, a student checks listings for the same model. A 2 year old one sells for more, a 6 year old one for less, and the rest fall roughly in between. Hers is 4 and a half years old, so she places it in the middle of that pattern and asks for a price to match. She has done regression in her head.
Where the example breaks: The student knows her scooter has a new battery and can adjust the price. A regression model only knows the inputs it was given, and if they leave out something important, its number will be off in a way it cannot notice.
How it actually works
A regression model maps features to a continuous value. It is the number-predicting half of supervised learning.
Linear regression
The classic model multiplies each feature by a weight, adds them up and adds a constant:
price = b + w1 × age + w2 × km. Training finds the weights. Google's glossary describes it as finding the line that best fits the labels to the features.What best fit means
Each example's gap between the line and the real value is an error. The usual loss function, mean squared error, squares those gaps and averages them, so big misses count far more than small ones. This is the least squares idea.
Finding the line
For a straight line there is a direct formula. For big models, gradient descent nudges the weights step by step until the loss stops falling.
Bending the line
Real patterns are rarely straight. Decision trees and neural networks can fit curves. More flexibility helps until the curve starts chasing noise, which is overfitting.
A confusing name
Logistic regression is used for classification: it predicts a probability that is then turned into a category. The word regression itself came from statistics, not from the idea of going backwards.
- Least squares first published
- 1805 by Adrien-Marie Legendre, then by Carl Friedrich Gauss in 1809, to work out the orbits of comets and planets
- Name
- Coined by Francis Galton in the 19th century, from children's heights "regressing" toward the average
- Output
- A number on a continuous scale
- Everyday uses
- Price estimates, demand forecasts, delivery time estimates, weather and energy forecasts
Questions people ask
What is the difference between regression and classification?
Regression predicts a quantity, such as a price or a temperature. Classification predicts a category, such as spam or not spam. If the possible answers form a fixed list, it is classification; if they sit on a scale, it is regression.
What is linear regression in simple terms?
Drawing the straight line that best fits a set of points, then using that line to predict new values. Each input gets a weight that says how much the output changes when that input goes up by one.
Why is it called regression?
Francis Galton used the word in the 19th century when he found that children of very tall parents tend to be closer to average height: their heights "regressed" toward the mean. The name stuck to the whole method of fitting such relationships.
Is logistic regression a regression or classification method?
It is used for classification. It predicts the probability of a yes, such as whether an email is spam, and a threshold turns that probability into a category. It is called regression for historical reasons.