Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Precision and recall

Also called precision, recall, F1 score

Precision asks how many of the model's alarms were right. Recall asks how many of the real cases it caught.

In plain words

Imagine a spam filter. It can be wrong in two different ways. It can flag a real message from your bank as spam, or it can let a spam message slip into your inbox. One number, like accuracy, mixes those up. Precision and recall pull them apart.

Precision looks at everything the filter flagged and asks: what share of it was really spam? High precision means few false alarms. Recall looks at all the spam that actually arrived and asks: what share did the filter catch? High recall means little slips through.

The two usually pull against each other. Make the filter stricter and it flags less, so precision rises but more spam gets through. Make it eager and it catches more spam, but also more real mail. Which matters more depends on the job: a cancer screening test wants high recall, while a filter that deletes mail needs high precision.

Everyday example

A fishing net

A fisherman wants pomfret. A fine net hauls in nearly every pomfret in the area, high recall, but also weeds and small fish, low precision. A net with big holes brings up mostly pomfret, high precision, but lets many escape, low recall. No single net is best; it depends whether he cares more about missing fish or sorting junk.

Where the example breaks: A net is fixed once woven. A model gives each case a score, and people choose the cut-off, the threshold, afterwards, so they can slide between high precision and high recall without retraining.

Go deeper

How it actually works

Both measures come from the four counts in a confusion matrix: true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN).

  1. Precision

    precision = TP ÷ (TP + FP). Of all the cases the model called positive, the share that really were. Google's course: use it when "it's very important for positive predictions to be accurate".

  2. Recall

    recall = TP ÷ (TP + FN). Of all the real positive cases, the share the model found. Also called sensitivity or the true positive rate. Use it when missing a case, a false negative, costs more than a false alarm.

  3. Why not just accuracy?

    If only 1% of cases are positive, a model that always says no is 99% accurate and useless. Precision and recall expose that, which is why they are used for imbalanced problems like fraud, disease or spam.

  4. The threshold trade-off

    A classification model usually outputs a score. Raising the threshold for saying yes tends to cut false positives (precision up) and add false negatives (recall down). Lowering it does the reverse.

  5. F1 score

    One number that balances both: the harmonic mean, F1 = 2 × precision × recall ÷ (precision + recall). It is high only when both are. In the picture, F1 = 2 × 0.75 × 0.6 ÷ 1.35 ≈ 0.67.

Precision
Also called positive predictive value: how many alarms were right
Recall
Also called sensitivity or true positive rate: how many real cases were caught
F-measure
Derived by C. J. van Rijsbergen in 1979, for judging search systems
In search
Information retrieval uses them too: are the results relevant, and were all relevant documents found?

Questions people ask

What is the difference between precision and recall?

Precision: of the things the model flagged, how many were correct? Recall: of the things that should have been flagged, how many did it find? Precision punishes false alarms; recall punishes misses.

Which is more important, precision or recall?

It depends on the cost of each mistake. Medical screening favours recall, because missing a disease is worse than a false alarm. A system that blocks payments or deletes messages favours precision, because false alarms hurt real users.

What is an F1 score?

A single number combining precision and recall, their harmonic mean. It ranges from 0 to 1 and is only high when both are high, so a model cannot score well by maximising one and ignoring the other.

Why is accuracy misleading?

When one class is rare, a model can be very accurate by always predicting the common class. If 1 in 100 transactions is fraud, saying never fraud is 99% accurate but catches nothing. Recall would be zero.

What is a false positive and a false negative?

A false positive is a false alarm: the model says yes when the truth is no, like real mail sent to spam. A false negative is a miss: it says no when the truth is yes, like spam reaching the inbox. See confusion matrix.

Further reading