In plain words
The word "bank" means one thing next to "river" and another next to "loan". To understand any word, we glance at the words around it and pick out the useful ones.
Attention gives an AI that same habit. For every word, the model gives each other word a score: how useful are you for understanding me? High scorers get blended in strongly, low scorers barely count. The scores for each word always add up to 100%.
It does this for every word in the text, all at once, and it repeats it in many layers. In a chatbot there is one rule: a word may only look back at words before it, never ahead, because the model writes one word at a time and the future has not been written yet.
Attention is the key idea inside the transformer, the design behind almost every modern large language model.
Everyday example
Reading a cricket score in a group chat
Someone types "What a catch!" in a group. To make sense of it, we scroll up and weigh each earlier message: the live score link matters a lot, the good-morning sticker hardly at all, yesterday's chat about dinner not at all. Every message gets a quick relevance score, and we build our understanding mostly from the high scorers. Attention does exactly this, for every word.
Where the example breaks: We decide what matters by understanding the situation. The model's scores come from patterns in numbers it learned during training; nobody wrote rules for them, and they do not always match what a person would pick.
How it actually works
Technically, attention compares vectors. The version used in transformers is called scaled dot-product self-attention.
Every word makes a query, a key and a value
Each token's vector is multiplied by three learned grids of numbers to make three new vectors. The query says what this word is looking for. The key says what it offers. The value is the information it will pass on.
Queries are matched against keys
The model takes one word's query and multiplies it with every word's key (a dot product). Similar directions give a high score. The scores are divided by a fixed number to keep them stable, then a function called softmax turns them into weights that add up to 1.
Values are blended by those weights
The word's new vector is the weighted sum of all the values. If "bank" gives "river" half its weight, half of what flows into "bank" comes from "river". The result carries context.
The grid and the mask
Doing this for all words gives a square grid of weights, one row per word. In chat models the top-right half is blocked, so no word can peek at later words. The 2017 paper calls this masking out illegal connections.
Many heads, many layers
Several attention heads run side by side, each learning its own kind of link, such as grammar or who-did-what. The original transformer used 8 heads per layer. Because every word is compared with every other, the work grows with the square of the text length, which is why context windows are costly.
- First described
- 2014, by Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, to help translation models look back at the source sentence
- Made central
- 2017, "Attention Is All You Need" from Google built a whole model from attention: the transformer
- The formula
softmax(QKᵀ / √d) V: match queries to keys, turn scores into weights, blend the values- Speed-up
- FlashAttention (Stanford, 2022) computes the same result with far fewer memory trips, helping long contexts
Questions people ask
What is the difference between attention and a transformer?
Attention is one mechanism: how each word weighs the others. A transformer is a full model design built around attention, plus other layers, stacked many times. Attention is the engine part; the transformer is the whole vehicle.
What does "Attention Is All You Need" mean?
It is the title of the 2017 Google paper that introduced the transformer. Earlier translation models combined attention with slower recurrent networks. The title claims attention alone is enough, and the results backed it up.
What are query, key and value in simple terms?
Think of a library. The query is what we are looking for, each book's key is the label on its spine, and the value is what is inside. Matching the query to labels decides which books we read, and how much of each.
What is self-attention?
Self-attention is attention within one piece of text: every word looks at the other words in the same passage. Cross-attention is when one text looks at another, for example a translation looking back at the original sentence.
Can attention show us why a model gave an answer?
Only partly. Attention weights show where information flowed, and researchers do inspect them. But many layers and heads interact, so one grid is not a full explanation. Researchers in explainability use attention as one clue among many.