In plain words
Teaching a chatbot what is helpful and what is harmful usually takes thousands of people rating its answers. That is slow, and the rules those people follow stay hidden in their heads.
Constitutional AI, developed by Anthropic, writes the rules down instead. The written set of principles is called a constitution. During training, the AI drafts an answer, reads the principles, points out what is wrong with its draft, and rewrites it. It then learns from the improved versions.
In a second stage, the AI compares pairs of answers against the principles and picks the better one. Those AI judgements replace most of the human ratings used in RLHF. The method is used to train Anthropic's Claude models.
Everyday example
A school's code of conduct
Instead of a teacher correcting every student's essay, a school gives students a written code: be respectful, cite sources, no copying. Each student checks their own draft against the code, marks the problems and rewrites. Then classmates compare pairs of essays using the same code. The rules are visible to everyone, and the checking scales.
Where the example breaks: Students understand the code and choose to follow it. The AI only learns patterns from the corrections, so it can still misread or miss a principle, and someone still has to decide what goes in the code.
How it actually works
Anthropic described the method in a December 2022 paper, "Constitutional AI: Harmlessness from AI Feedback".
Write the principles
The only human input to the harmlessness training is "a list of rules or principles". Anthropic's 2023 version drew on sources including the UN Declaration of Human Rights, other companies' platform rules and DeepMind's Sparrow principles.
Stage 1: critique and revise
The model answers a difficult prompt, then is asked to critique its own answer against one of the principles, and to rewrite it. The model is then fine-tuned on the revised answers.
Stage 2: AI feedback
The model generates two answers, and an AI judges which better follows the principles. Those judgements train a preference model, a kind of reward model, which then guides reinforcement learning. This is called RL from AI Feedback, or RLAIF.
Explain, don't just refuse
The paper's goal was an assistant that is "harmless but non-evasive": one that explains why it will not help with something harmful, instead of simply refusing. Both stages can use chain of thought reasoning, which makes the AI's judgements easier to inspect.
A longer constitution in 2026
In January 2026 Anthropic published a new constitution for Claude, released for anyone to reuse. Where the old one was a list of standalone principles, the new one explains its reasoning, and sets an order of priority when values conflict: broadly safe, broadly ethical, following Anthropic's guidelines, then genuinely helpful.
- Developed by
- Anthropic; paper by Yuntao Bai and colleagues, December 2022
- Also called
- RLAIF (reinforcement learning from AI feedback) for the second stage
- Public input experiment
- October 2023: about 1,000 members of the American public helped choose principles, with the Collective Intelligence Project
- Licence
- Anthropic released its January 2026 constitution under Creative Commons CC0
Anthropic itself says Claude's outputs "might not always adhere to the constitution's ideals". A written constitution makes the goals visible; it does not guarantee the behaviour.
Questions people ask
What is in Claude's constitution?
As of October 2026, it is a long document published by Anthropic in January 2026 that describes the values and character it wants Claude to have and explains why. It asks Claude to put broad safety first, then ethics, then Anthropic's guidelines, then helpfulness, when these conflict.
Is Constitutional AI the same as RLHF?
Related but different. RLHF relies on many people rating answers. Constitutional AI replaces most of those ratings with AI judgements guided by written principles, and adds a self-critique stage. Human effort shifts from rating answers to writing and refining the principles.
Who writes the constitution?
Anthropic's staff write it. In 2023 Anthropic also ran an experiment where about 1,000 members of the American public helped choose principles through an online discussion platform. Who should set AI values is an open question in AI governance.
Does a constitution make an AI safe?
It helps make the intended behaviour explicit and easier to check, but the model learns from examples and can still make mistakes or be jailbroken. It is one tool within wider alignment and safety work, alongside testing and red teaming.