Logistic regression: what it is, explained simply

In plain words

Lots of questions have a yes or no answer: will it rain, is this email spam, will this customer pay back the loan? Logistic regression answers them with a probability, a number between 0 and 1, rather than a flat yes or no.

It works in two steps. First it gives each clue a weight and adds them into a single score. Clues that point to yes push the score up, clues that point to no pull it down. Then a smooth S-shaped curve squeezes any score into the range 0 to 1. A big positive score lands near 1, a big negative one near 0.

To get a decision, you pick a cut-off, often 0.5: above it means yes. The weights are learned from past examples with known answers, which makes it supervised learning. Despite its name, it is mostly used for classification, and it is a common first model to try: a baseline that fancier models must beat.

Everyday example

Should you carry an umbrella?

Before leaving for work you glance outside. Dark clouds count strongly towards rain, sticky air counts a little, and a strong breeze counts slightly against. You add it up and feel "fairly likely, maybe 8 in 10". Above your personal cut-off of 5 in 10, so the umbrella goes in the bag. Logistic regression does this sum with learned weights, then turns it into an exact probability.

Where the example breaks: You weigh clues by gut feeling. Logistic regression learns each weight from many past cases, and it assumes each clue adds its push separately, so it misses combinations like "clouds matter only when it is hot" unless someone adds that combination as a clue.

How it actually works

Logistic regression is a linear model with an S-shaped function on its output.

A weighted sum of features

Each input feature x gets a learned weight w, and there is a bias b. The model computes z = b + w₁x₁ + w₂x₂ + ... This z is the log-odds: the logarithm of the odds of a yes.

The sigmoid squeezes it

The score goes through the sigmoid function, 1 / (1 + e^(−z)). Whatever z is, the output stays strictly between 0 and 1. A score of 0 gives 0.5, a score of 2 gives about 0.88, and a score of −10 gives almost exactly 0.

Learning the weights with log loss

Training adjusts the weights with gradient descent to minimise log loss, a loss function that punishes confident wrong answers very heavily. Regularisation keeps weights from growing without limit when the data has many features.

A threshold makes the call

A person, not the training, chooses the cut-off that turns a probability into a yes or no. Raising it means fewer false alarms but more misses, a trade-off measured with precision and recall.

More than two classes

Multinomial logistic regression handles several categories by giving each one a score and turning the scores into probabilities that add up to 1. That step, called softmax, is also how many neural networks turn their final scores into probabilities.

Roots

Pierre François Verhulst used the logistic curve to model population growth in papers from 1838 to 1847, and named it "logistic" in 1845

"Logit" coined

By Joseph Berkson in 1944

Developed by

Statistician David Cox, including a 1958 paper on regression for yes or no data

Output

A probability between 0 and 1, turned into a category by a threshold

Questions people ask

Why is it called regression if it is used for classification?

Because it predicts a number, the probability, in the way regression does, and the name comes from statistics. A threshold then turns that number into a category, which is why it is used as a classification method.

What is the difference between linear and logistic regression?

Linear regression predicts any number, such as a price, with a straight-line formula. Logistic regression uses the same kind of weighted sum but passes it through an S-curve, so the result is always a probability between 0 and 1.

What is the sigmoid function?

An S-shaped curve, 1 / (1 + e^(−x)), that squashes any number into the range 0 to 1. Very negative inputs come out near 0, very positive ones near 1, and 0 comes out as exactly 0.5.

Is logistic regression machine learning?

Yes. It began in statistics, but it learns its weights from examples, which is what machine learning means. It is often the first baseline a team tries before more complex models.