Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

AI bias

Also called algorithmic bias, bias

AI bias is when an AI system treats some groups of people unfairly, usually because it learned from data that was already skewed.

Safety and societyEveryday

In plain words

An AI system learns by studying lots of past examples. If those examples are lopsided, the system picks up the lopsidedness and repeats it, often without anyone noticing.

Say a company trains an AI on ten years of its own hiring decisions. If it mostly hired men in the past, the AI can learn that looking like past hires means a good candidate. Then it scores an equally skilled woman lower. Nobody told it to. It simply copied the pattern.

Bias can also come from who is missing in the data, from what the system was asked to predict, or from how it is used. The fix starts with measuring: checking how well the system works for each group, not just on average.

Everyday example

A cook who learned from one family

A cook learns every recipe from one family that loves very spicy food. Later he cooks for a wedding with guests from all over the country. He means well, but half the guests find the food too hot. His taste is not wrong. It is just built from a narrow set of examples. An AI trained on narrow data serves everyone the same way.

Where the example breaks: The cook can taste the food and notice the complaints. An AI system does not notice its own unfairness. Someone has to measure it deliberately, group by group.

Go deeper

How it actually works

Bias can enter at every stage, from collecting data to using the result.

  1. Skewed or missing data

    If some groups are rare in the training data, the model sees fewer examples of them and makes more mistakes on them. The 2018 Gender Shades study found commercial face analysis tools misclassified darker-skinned women far more often than lighter-skinned men.

  2. History baked into labels

    Data often records past human decisions, such as who was hired or who got a loan. A model trained to copy those decisions copies their unfairness too. Removing a field like gender does not always help, because other fields (a college name, a postcode) can stand in for it.

  3. The wrong target

    Sometimes the model predicts something that is only a rough stand-in for what we care about. Predicting "past arrests" is not the same as predicting "crime", because who gets arrested depends on where police look.

  4. Measuring it

    Teams compare error rates and outcomes across groups. There are several fairness tests, and a well-known 2016 result proved that, outside special cases, a risk score cannot pass three of the most common ones at the same time. Choosing which to prioritise is a human judgement.

  5. Reducing it

    Common fixes: collect more balanced data, test each group separately before launch, adjust the model or its thresholds, keep a human in the loop for high-stakes decisions, and keep monitoring after launch.

Gender Shades (2018)
Error rates up to 34.7% for darker-skinned women, at most 0.8% for lighter-skinned men, across commercial gender classifiers
A hiring case
Reuters reported in October 2018 that Amazon had dropped an experimental hiring tool that marked down CVs containing the word "women's"
A risk-score dispute
ProPublica's 2016 analysis of the COMPAS tool in Florida found Black defendants who did not reoffend were wrongly flagged high risk almost twice as often as white defendants. The maker disputed the analysis
In law
The EU AI Act treats AI used in hiring, credit and education as high-risk. Those duties now start on 2 December 2027 (as of October 2026)

Bias is not only about people. A medical model trained in one country's hospitals can also work worse in another's.

Which kind of fairness should an AI aim for?

One view

Scores mean the same

A score should mean the same thing for everyone. If two people get a 7 out of 10 risk score, they should have the same real chance of the outcome, whatever group they belong to.

The other view

Equal mistakes

Wrong calls should fall evenly. Innocent people in one group should not be wrongly flagged more often than innocent people in another, even if scores stay accurate on average.

Most likely: Both matter, and maths shows they usually cannot both be fully met when groups differ in their base rates. Most experts say the choice should be made openly, for each use, by people accountable for it.

Questions people ask

Can AI be biased if it has no opinions?

Yes. It has no feelings, but it learns patterns from data people produced. If the data reflects unequal treatment or leaves groups out, the system repeats that. Bias here means a systematic, unfair difference in results, not a personal belief.

Is it enough to remove gender or caste from the data?

Usually not. Other details can act as stand-ins, such as a name, a college, a neighbourhood or a language. A model can rebuild the hidden attribute from them. Testing the results group by group is the reliable way to find out.

Do chatbots have bias too?

Yes. A large language model learns from huge amounts of internet text, so it can pick up stereotypes about jobs, regions or religions. Makers try to reduce this with testing and feedback training, but it is not fully solved.

Is a biased AI worse than a biased human?

It depends. An AI applies the same pattern to millions of decisions, so a flaw spreads fast. But it is also easier to audit: you can test it on thousands of cases and fix it, which is hard to do with a person.

How can you tell if an AI decision was biased?

Often you cannot tell from one decision. Ask the organisation how the system was tested and whether a human can review it. Some laws, such as the EU AI Act for high-risk uses, require testing and records.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.