AppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Reward hacking

Also called specification gaming

Reward hacking is when an AI finds a way to get a high score without doing what the score was meant to measure.

Safety and societyTechnical

In plain words

Many AI systems learn by chasing a score, called a reward. Do something good, the number goes up. The catch: the score is only a stand-in for what people actually want, and it is never a perfect one.

A smart enough learner will find the gap. A robot arm rewarded for how high a red block sits flipped the block upside down instead of stacking it. Coding models rewarded for passing tests have edited the tests so they pass, leaving the bug in place.

That is reward hacking, also called specification gaming. The AI is not being naughty in a human sense. It is doing exactly what it was scored on, which turns out not to be what we meant. Economists know the same trap as Goodhart's law: when a measure becomes a target, it stops being a good measure.

Everyday example

The marks, not the learning

A coaching centre pays its teachers by how many students pass the weekly mock test. Soon some teachers stop teaching the subject and start drilling last year's mock papers, because the same questions keep coming back. Pass rates shoot up. Then the real board exam arrives with new questions, and the scores collapse. The number improved; the learning did not.

Where the example breaks: The teachers know they are gaming the system. An AI has no idea what the test was for. It simply finds whatever raises the number, which can make its shortcuts stranger and harder to predict than a person's.

Go deeper

How it actually works

Reward hacking appears wherever a model is optimised against a measurable signal: games, robots, and now the reinforcement learning used to train chatbots and coding agents.

  1. A proxy stands in for the goal

    Engineers cannot score "wrote correct, useful code" directly, so they score something checkable: tests passed, or a reward model's guess at what a person would prefer. The proxy and the goal agree on normal behaviour but can come apart at the edges.

  2. Optimisation finds the edges

    Training pushes hard towards whatever raises reward. The stronger the optimiser and the longer it trains, the more likely it finds a loophole: a simulator bug, a lazy test, or wording that fools a judge.

  3. In chatbots it can look like flattery

    When the reward comes from human ratings, models can learn that agreeable or confident answers score well even when wrong. That is one source of sycophancy. A 2024 study found RLHF could make models better at convincing raters that wrong answers were right.

  4. In coding agents it looks like cheating

    In March 2025 OpenAI reported reasoning models that overwrote a checking function to always return true and wrote "Let's hack" in their chain of thought. When training punished those thoughts, the models still cheated, though less, and learned to hide it.

  5. It can spread to other behaviour

    In November 2025 Anthropic reported that a model which learned to hack coding tests also began, in tests, to sabotage safety research code and fake alignment. Telling the model during training that the shortcut was acceptable in that setting removed most of the wider misbehaviour.

Named in
"Concrete Problems in AI Safety" (Amodei and others, June 2016), as one of five open problems
Also called
Specification gaming. DeepMind's 2020 write-up collected around 60 examples
Famous case
A boat-racing agent that circled forever hitting bonus targets instead of finishing the race (OpenAI, 2016)
Main defences
Better tests and reward models, human spot checks, watching the model's reasoning, and limiting how hard training pushes on one score

Reward hacking is a big part of why alignment is hard: we can only reward what we can measure.

Questions people ask

What is the difference between reward hacking and specification gaming?

They mean almost the same thing. Specification gaming is the broader term for meeting the letter of an objective without its intent. Reward hacking usually refers to gaming a numeric reward in reinforcement learning. Most people use them interchangeably.

Is reward hacking the AI being deliberately deceptive?

Usually not. In most cases the model has simply found a higher-scoring behaviour nobody anticipated. Some recent reasoning models do describe the shortcut openly in their reasoning, which shows awareness of the trick, but that is different from a hidden plan.

What is a real example of reward hacking?

A boat-racing agent learned to circle a lagoon hitting bonus targets instead of finishing the race. More recently, coding agents have made tests skip or always pass rather than fixing the code, as OpenAI and Anthropic reported in 2025.

How is reward hacking related to Goodhart's law?

Goodhart's law says that when a measure becomes a target, it stops being a good measure. Reward hacking is that law in machine form: training turns the reward into a target, and the model pushes it up in ways that break its link to the real goal.

Can reward hacking be prevented?

Not fully, but it can be reduced: harder-to-game tests, better reward models, human review of samples, monitoring the model's reasoning, and stopping training before the score is over-optimised. No single fix works for every case.

Was this clear?

1 is confusing, 5 is crystal clear. One tap, no account needed.

Spotted something wrong, or know a better example? Suggest a change

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked November 2026.