Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Data poisoning

Also called poisoning attack

Data poisoning is slipping bad examples into the data an AI learns from, so the finished model behaves wrongly, often only in hidden, specific situations.

Safety and societyTechnical

In plain words

An AI model learns from its training data. Large models learn from huge collections of text and images gathered from the web, much of which nobody checks by hand.

That creates an opening. If someone can get tampered examples into that data, the model learns from them too. This is data poisoning. It can make a model worse overall, push it towards a false belief, or plant a hidden "backdoor": the model behaves normally until it sees a particular trigger, then does something else.

Research suggests attackers may need surprisingly few poisoned examples. That is why AI builders now treat their data supply like a food supply: know where it came from, test it, and watch for anything odd.

Everyday example

One bad batch at the spice mill

A restaurant chain buys its garam masala from one big mill. If someone tampers with a single batch at the mill, every branch that uses it serves the problem, and nobody in the kitchens notices because the dish still looks right. Data poisoning works at the source in the same way: tamper with the data once, and every copy of the trained model carries the flaw.

Where the example breaks: Spoiled masala usually tastes off. A poisoned model can pass every normal test and only misbehave on a trigger the attacker chose, which is what makes backdoors hard to find.

Go deeper

How it actually works

Security researchers count poisoning as an integrity attack: it corrupts what the model learns, rather than attacking it after it is built.

  1. Where poison can enter

    In the huge web data used for pre-training, in smaller datasets used for fine-tuning, or in the documents a system looks up while answering, as in RAG. OWASP's 2025 list of top risks for language model apps covers all three.

  2. What it can do

    Broad poisoning lowers quality or adds bias. Targeted poisoning makes the model wrong about one thing. A backdoor keeps the model normal until a trigger appears. In a 2024 study, researchers deliberately built backdoored test models and found standard safety training did not remove the hidden behaviour.

  3. How little it can take

    A 2023 study estimated it would have cost about 60 US dollars to poison 0.01% of two popular image datasets built from web links. In 2025, Anthropic, the UK AI Security Institute and the Alan Turing Institute found about 250 poisoned documents planted a simple backdoor in models of every size they tested, from 600 million to 13 billion parameters.

  4. Defences

    Recording where every piece of data came from, vetting data suppliers, filtering and scanning for unusual examples, keeping versions so changes can be rolled back, red teaming for hidden triggers, and watching the model's behaviour after release. The 2025 researchers note defenders have an advantage: poison must be placed before training, and can then be searched for.

Early case
Microsoft's Tay chatbot, March 2016, learned from users who deliberately fed it offensive messages and was suspended within 16 hours
Small numbers
About 250 documents were enough for a narrow backdoor across model sizes tested (October 2025). Whether this holds for bigger models and more harmful behaviour is unknown
Used by artists too
Nightshade (University of Chicago, 2023) alters images so models that scrape them without permission learn the wrong things
Listed risk
LLM04 in the OWASP Top 10 for LLM Applications 2025

We explain the idea and the defences only. We do not publish how to carry out these attacks.

Questions people ask

What is data poisoning in simple words?

Tampering with the examples an AI learns from, so the finished model learns something wrong. It might get worse overall, believe a falsehood, or carry a hidden trigger that makes it misbehave in one specific situation while seeming normal otherwise.

What is a backdoor in an AI model?

A hidden behaviour planted during training that only appears when a particular trigger, such as a word or pattern, shows up in the input. The rest of the time the model behaves normally, so ordinary testing may not catch it.

Can someone poison ChatGPT by chatting with it?

Not directly. Your chats do not instantly change the model. Some companies may use chats in later training, with filtering, so in principle bad data could reach it at scale. That is why labs filter and check training data. Tay in 2016 was different: it learned from users live.

Is data poisoning the same as prompt injection?

No. Poisoning corrupts what a model learns during training, so the flaw is inside the model. Prompt injection hides instructions in the text a model reads while it is working. Both are listed in OWASP's top risks for language model apps.

Is it legal for artists to poison AI training data?

It is debated, and laws differ by country and use. Nightshade's makers present it as a last defence for artists against web scrapers that ignore opt-out requests. We are not lawyers; see AI and copyright for the wider debate.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.