In plain words
In 2014, researchers took a photo of a panda that an image model recognised correctly. They added a faint layer of noise, too subtle for a person to notice. The picture still looked exactly like a panda. The model now said "gibbon", and was more confident than before.
That doctored photo is an adversarial example. It is not random noise. The change is carefully chosen to push the model across the line between one answer and another, using weak spots in how it sees.
It matters because AI is used where mistakes are costly: reading road signs, screening X-rays, spotting fraud, filtering harmful posts. In one study, a few black and white stickers on a real stop sign made a research classifier read it as a speed limit sign in most test images. Building models that resist such tricks is an active area of AI safety.
Everyday example
The note-counting machine
A bank's note-counting machine checks each note in a split second, looking at a few features. A cashier would spot a strange note by eye. Now imagine someone studies exactly which features the machine checks, then changes a note in a way invisible to the cashier but which flips the machine's decision. That is an adversarial example: tuned to the machine, unnoticed by people.
Where the example breaks: A counting machine checks a few known features. A neural network's weak spots are spread across millions of numbers and are not designed in, which is why they are hard to find and hard to fix.
How it actually works
Security researchers call this an evasion attack: fooling a finished model at the moment it is used, without touching its training.
Why models are fooled
A neural network decides by adding up many tiny signals. Goodfellow and colleagues argued in 2014 that because models behave in a largely linear way, many tiny nudges across all the pixels add up to a big change in the output. People do not see those nudges at all.
They often transfer
The 2013 paper that first described the effect found a change made to fool one network often fooled another network trained separately. That means an attacker does not always need the exact model.
Beyond images
The same idea applies to audio, text and language models. Some jailbreaks use automatically found strings of odd text that push a chatbot past its safety training, a kind of adversarial example for words.
Defences
Adversarial training adds fooled examples to training so the model learns to resist them. Certified methods can prove a model will not change its answer for changes below a set size, at some cost in accuracy. Other layers include checking inputs, combining several models and keeping a human in the loop for high-stakes calls. No defence is complete.
An arms race
Several proposed defences were later broken by stronger attacks. OpenAI wrote in 2017 that even the best early defences "can easily be broken by giving more computational firepower to the attacker", and that a defence has to hold for every possible input, not just the ones tested.
- First described
- December 2013, "Intriguing properties of neural networks" (Szegedy and colleagues)
- Famous example
- Panda at 57.7% confidence becomes "gibbon" at 99.3% after a tiny change (Goodfellow, Shlens and Szegedy, 2014)
- In the physical world
- Black and white stickers on a real stop sign made a research classifier read it as a Speed Limit 45 sign (2017 study)
- Classed as
- An evasion attack in NIST's 2025 guide to adversarial machine learning
We explain the idea and the defences only. We do not publish how to craft attacks.
Questions people ask
What is an adversarial attack in simple words?
Deliberately changing an input, such as a photo, a sound or some text, in a tiny way chosen to make an AI model give a wrong answer. To a person the input looks or sounds the same. The model's answer changes completely.
Why can a tiny change fool an AI but not a person?
People recognise things using shape, context and common sense. A model adds up thousands of small numeric signals, and many tiny nudges in the same direction can add up to a big shift. The change is chosen to target exactly those signals, which our eyes ignore.
Is an adversarial example the same as data poisoning?
No. Data poisoning tampers with the training data, so the flaw is built into the model. An adversarial example fools a model that is already trained, at the moment it is used. NIST calls these poisoning and evasion attacks.
Can adversarial examples fool self-driving cars?
In a 2017 study, stickers on a real stop sign fooled a research image classifier in lab photos and in most frames filmed from a moving car. That was a research model, not a specific car on the road. It shows why makers of safety-critical systems test against such tricks.
Can adversarial examples be used for good?
Yes. Researchers and companies create them on purpose to test how robust a model is before release, as part of red teaming. Adversarial training then feeds them back into training so the model learns to resist them.
Further reading
- Explaining and Harnessing Adversarial ExamplesarXiv, 2014
- Intriguing properties of neural networksarXiv, 2013
- Attacking machine learning with adversarial examplesOpenAI, 2017
- Robust Physical-World Attacks on Deep Learning Visual ClassificationarXiv, 2017
- Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and MitigationsNIST, 2025