In plain words
Every AI model has a context window: the amount of text it can see at once. A few years ago that was a few pages. As of November 2026, the main models from OpenAI, Anthropic and Google accept about a million tokens, roughly 750,000 English words.
That changes what you can ask. You can paste a whole contract bundle, a year of meeting notes or an entire codebase, and ask questions across all of it, instead of feeding it in pieces.
But fitting it in is not the same as reading it well. Tests show models are best at using facts near the start or end of a long input, and miss more in the middle. Accuracy also tends to slip as inputs get longer, even on simple tasks. Long inputs are slower and cost more too, since you pay for every token each time.
Everyday example
A lawyer handed the whole case file
A junior lawyer gets three thick files and one question: "Did the builder ever promise a June handover?" Having all the files on the desk beats fetching one page at a time. But a tired reader skims the middle of the pile and can miss the one email that matters. Long-context models have the same strength and the same weak spot.
Where the example breaks: A lawyer gets tired. A model doesn't, but it still spreads its attention thinly over a huge input, so details in the middle can get less weight. More text can mean more distraction.
How it actually works
Making long context work takes changes to training and to how models are run, because cost and memory grow with every extra token.
Why length is hard
In a transformer, each token pays attention to every earlier token, so work grows roughly with the square of the length. The model also has to keep notes on every token in its KV cache, which takes memory for each token you add.
Teaching models to handle longer text
Models learn word order from position signals (positional encoding). Researchers found ways to stretch these signals, so a model trained on shorter text can handle much longer text after brief extra training.
Testing it: needles and beyond
The "needle in a haystack" test hides one fact in a long text and asks for it. Most long-context models score nearly perfectly. Harder tests, which need several facts combined, show many models fall short of their advertised length.
Using it well
Put the question after the long material, give the model only what is relevant, and reuse long inputs with prompt caching to cut cost. For huge, changing collections, retrieval is often still the better tool.
- The jump
- Google's Gemini 1.5 offered 1 million tokens in preview in February 2024. Its report described near-perfect recall of single facts in research tests up to 10 million
- Typical size, November 2026
- About 1 million tokens for flagship models from OpenAI, Anthropic and Google
- Lost in the middle
- A 2023 Stanford-led study found answers were best when the key fact sat near the start or end
- Effective vs advertised
- In a 2024 NVIDIA test of 17 models that all claimed 32,000 tokens or more, only about half held up well at 32,000
Questions people ask
How much is a 1 million token context?
Roughly 750,000 English words. Google's guide puts it at about 8 average-length novels, or around 50,000 lines of code. Many other languages use more tokens per word, so the same window holds less of them.
Is long context better than RAG?
They solve different problems. Long context is simplest when the material fits and you need all of it. RAG is better for huge or changing collections, since it fetches only the relevant parts, which is cheaper and often more accurate.
Does the AI actually read everything in a long context?
It processes every token, but it does not use all of it equally well. Facts buried in the middle are more likely to be missed, and accuracy tends to drop as inputs grow. For important questions, point the model to the right section.
Why is long context expensive?
Providers charge per token, and every token you send is counted on every request. A 500,000-token document sent with ten questions is billed ten times, unless the provider lets you cache it. Long inputs also take longer to process.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change
Further reading
- Lost in the Middle: How Language Models Use Long Contexts (Liu et al.)arXiv, 2023
- RULER: What's the Real Context Size of Your Long-Context Language Models? (Hsieh et al.)arXiv, 2024
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextarXiv, 2024
- Long contextGoogle AI for Developers, 2026
- Context Rot: How Increasing Input Tokens Impacts LLM PerformanceChroma, 2025