In plain words
Computers can't compare meanings directly. They can compare numbers. An embedding is how AI bridges that gap: it converts something like the word "mango" into a long list of numbers, often hundreds or thousands of them.
The clever part is how those numbers are chosen. They are learned, so that "mango" lands near "banana" and far from "bus". Think of each embedding as a point on a giant map of meaning, with far more directions than north, south, east and west.
Once meaning is a position, closeness becomes something a computer can measure. That is how AI search finds a document about "low back pain" when you typed "my lower back hurts", even though few words match.
Everyday example
Arranging a kirana store
A good shopkeeper doesn't stock items alphabetically. Rice sits next to dal, biscuits next to namkeen, soap next to shampoo. You can walk in, find the soap, and know the shampoo is nearby. Embeddings arrange words and ideas the same way, so related things sit together.
Where the example breaks: A store has one floor and a person decides the layout. An embedding space has hundreds of dimensions, and the layout is learned from data, so it also picks up the biases in that data.
How it actually works
An embedding is a vector: an ordered list of numbers. Models learn them so that useful relationships become geometry.
Learned, not hand-written
Nobody decides what each number means. During training, the model adjusts the vectors so that items appearing in similar contexts get similar vectors. Early methods like word2vec did this by predicting nearby words.
Distance means similarity
To compare two embeddings, systems usually measure the angle between them (cosine similarity). A small angle means similar meaning. This calculation is fast, even across millions of items.
Directions carry meaning
Relationships can show up as consistent directions. In word2vec, the step from "man" to "woman" pointed roughly the same way as "king" to "queen", so king minus man plus woman landed near queen.
Embeddings as a product
Separate embedding models turn whole sentences or documents into one vector. These power semantic search, recommendations, duplicate detection and retrieval-augmented generation, usually stored in a vector database.
- Breakthrough
- word2vec, by Tomas Mikolov and colleagues at Google, 2013
- Typical size
- Hundreds to a few thousand numbers per item. OpenAI's 2024 embedding models output 1,536 or 3,072 by default
- Works for
- Words, sentences, documents, images, audio, products, even users
- Closely related
- Latent space, the space these vectors live in
Questions people ask
What is the difference between an embedding and a vector?
A vector is any list of numbers. An embedding is a vector that was learned to represent the meaning of something. Every embedding is a vector, but not every vector is an embedding.
What are embeddings used for?
Search by meaning, recommendations ("people who liked this also liked"), grouping similar documents, spotting duplicates, and giving chatbots the right documents to read through RAG.
Do embeddings work for languages other than English?
Yes. Multilingual embedding models place sentences with the same meaning close together across languages, so a Hindi question can find an English document that answers it.
Can you read what each number in an embedding means?
Usually not. The individual numbers are not labelled with meanings. Researchers can sometimes find directions that track ideas like gender or tense, but most dimensions have no simple meaning. That is part of the explainability challenge.