In plain words
Before a new AI model is released, its makers want to know the worst it can do. So they hire people whose job is to break it: to coax out dangerous advice, private data, biased answers or rule-breaking. This friendly attack team is called a red team.
The name comes from Cold War military exercises, where the red team played the enemy and the blue team defended. Computer security borrowed it, and now AI does too.
Red teamers include security experts, doctors, chemists, lawyers and ordinary users, because harms can be technical or human. What they find goes back to the builders, who retrain the model or add guardrails, and then test again.
Everyday example
The friend who reads your speech first
Before giving a wedding speech, you ask your bluntest friend to listen and pick holes. She points out a joke that would offend an uncle, a name you got wrong and a story that runs too long. You fix them before the big day. A red team is that blunt friend for an AI model, hunting for what could go wrong before the public sees it.
Where the example breaks: Your friend can hear the whole speech. Red teamers can only try a tiny share of the endless things people might ask an AI, so passing red teaming lowers risk but cannot prove a model safe.
How it actually works
AI red teaming mixes human creativity with automated testing.
Decide what to look for
Teams list the harms that matter for this model: help with weapons or cyberattacks, jailbreaks, prompt injection, leaking private data, bias, or dangerous behaviour by AI agents.
Human experts attack
Specialists probe the model in their own field. OpenAI set up a paid Red Teaming Network in 2023 covering areas from biology and law to persuasion and child safety. Anthropic released nearly 39,000 human red-team attacks from 2022 for others to study.
AI attacks AI
Humans are slow, so language models are also used to generate thousands of test attacks automatically. DeepMind showed this in 2022, finding tens of thousands of offensive replies in a large chatbot.
Fix and re-test
Findings feed back into training, filters and policies, and the fixed model is attacked again. Many companies publish a summary of what testing found alongside a new model.
Known limits
Red teaming finds problems; it cannot show there are none. The 2026 International AI Safety Report notes that tests before release may not predict real-world behaviour, and some models can tell when they are being tested.
- Origin of the term
- US military and RAND Corporation exercises in the Cold War, where red stood for the opposing side
- A big public exercise
- DEF CON 31, Las Vegas, August 2023: 2,244 people sent 164,208 messages to models from eight companies
- Who does it
- AI companies, paid outside experts, government safety institutes and independent researchers
- Related
- Benchmarks measure abilities on fixed tests; red teaming searches openly for failures
Questions people ask
What does a red team do in AI?
It tries to make an AI system fail on purpose: give harmful instructions, reveal private data, show bias, or break its rules. Everything it finds is reported to the builders so they can fix it before or after launch.
What is the difference between red team and blue team?
The red team attacks, the blue team defends. In AI, the blue team is the builders and safety engineers who patch the problems the red team finds, then the cycle repeats.
Can anyone become an AI red teamer?
Often, yes. Companies recruit experts in many fields, not just computing, and OpenAI said prior AI experience was not required for its 2023 network. Public events and bug bounty programmes also let outsiders take part.
Is red teaming the same as hacking?
It uses similar thinking, but it is authorised and done to improve safety. For AI it is mostly about language: clever questions and scenarios rather than breaking into computers, though security testing is part of it.