Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Retrieval-augmented generation

Also called RAG, grounding, chat with your documents

RAG lets an AI look up the relevant pages in a set of documents first, then write its answer from what it found.

Using AICurious

In plain words

A language model only knows what was in its training data. It has never seen your company's HR policy, last week's news or the manual for your washing machine. Ask about them and it will either refuse or guess.

Retrieval-augmented generation, or RAG, fixes this with an extra step. Before the AI answers, a search system finds the few passages most related to your question. Those passages are pasted into the prompt, and the AI writes its answer from them, often with a note of where each fact came from.

It is how "chat with your documents" features, company help bots and AI search answers with citations usually work. To update what the AI knows, you just update the documents. No retraining needed.

Everyday example

An open-book exam

In a closed-book exam, you answer from memory and may misremember. In an open-book exam, you first flip to the right chapter, read the paragraph, then write your answer from it. RAG turns the AI's exam into an open-book one: find the right pages, then answer.

Where the example breaks: A student picks the right chapter using understanding. RAG's search can fetch the wrong passage, or miss the right one, and then the AI answers confidently from the wrong page.

Go deeper

How it actually works

A RAG system has two halves: a retriever that finds text and a generator, the language model, that writes the answer.

  1. Prepare the library

    Documents are split into short chunks, a few paragraphs each. Each chunk is turned into an embedding, a list of numbers that captures its meaning, and stored, usually in a vector database.

  2. Search by meaning

    Your question is turned into an embedding too. The system finds chunks whose numbers are closest, so "leave days" can match a page that says "annual holiday entitlement". Many systems also add ordinary keyword search.

  3. Build the prompt

    The top few chunks are inserted into the prompt with an instruction such as "Answer using only the passages below and cite them. If the answer isn't there, say so."

  4. Generate the answer

    The model writes a reply grounded in those passages. Because it read real text, hallucinations drop, though they do not disappear: the model can still misread, or the search can fetch the wrong chunk.

Named in
"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", Patrick Lewis and colleagues, May 2020
Where
Facebook AI, now Meta AI
Key promise
In Meta AI's 2020 blog post: "we control what it knows simply by swapping out the documents"
Common uses
Company knowledge bots, customer support, "chat with your PDF", AI search with citations, deep research tools

RAG is one part of the wider craft of choosing what a model sees, called context engineering.

Questions people ask

What is RAG in simple terms?

Search first, then answer. The system finds the most relevant passages in a set of documents, adds them to the AI's prompt, and the AI writes its reply from them. It lets an AI answer about material it was never trained on.

What is the difference between RAG and fine-tuning?

RAG hands the model relevant text at question time, so knowledge can be updated by editing documents. Fine-tuning retrains the model's weights on examples, which is better for teaching a style or skill than for keeping facts current.

Does RAG stop hallucinations?

It reduces them a lot but does not stop them. If the search returns the wrong passage, or the model misreads it, you can still get a confident wrong answer. Good RAG systems show their sources so you can check.

Does RAG always need a vector database?

Not always. Small collections can be searched with simple tools, and many systems mix keyword search with meaning-based search. A vector database becomes useful when there are many documents and you want fast search by meaning.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.