By Quick answer: What is RAG (retrieval-augmented generation)?
- Simple version — an open-book exam for AI.
- Steps — retrieve relevant passages, then generate an answer from them.
- Why — current, private and verifiable answers without retraining.
- Pieces — chunking, embeddings, search, prompt, citations.
- Quality — retrieval quality decides answer quality; evaluate both.
Key takeaways
- Most 'chat with your documents' products are RAG systems.
- Bad retrieval leads to confident wrong answers — measure it.
- Hybrid search (keywords + meaning) often beats either alone.
- Citations let users verify and build trust.
- RAG and fine-tuning solve different problems; many systems use RAG first.
Imagine two students taking an exam. One has to answer from memory. The other can bring the textbook, find the right page, and write the answer with the page number. The second student is less likely to make things up — and you can check their work. That's retrieval-augmented generation (RAG): letting an AI look things up before it answers. The idea was formalised in a 2020 research paper by Patrick Lewis and colleagues1.
The five-year-old version
- You ask a question.
- The system looks in its books for the pages that talk about it.
- It reads those pages.
- It answers using those pages — and tells you which pages it used.
The builder's version
| Stage | What happens | Key decisions |
|---|---|---|
| Ingest | Documents are cleaned and split into chunks | Chunk size and overlap; keep headings and metadata |
| Embed | Each chunk is turned into an embedding (a vector of meaning) | Embedding model; language support |
| Index | Embeddings stored in a vector index (plus keyword index) | Hybrid search; filters by user or permission |
| Retrieve | The question is embedded; similar chunks are found | How many results; re-ranking |
| Generate | Retrieved chunks + question go to the model | Instructions to answer only from sources; citations |
Why RAG beats asking the model alone
Current
Update the documents and answers update — no retraining.
Private
Use your own internal knowledge securely, with permissions.
Verifiable
Citations let users check the source.
Cheaper
Usually far simpler than training or fine-tuning a model.
Where RAG goes wrong
- Bad chunks — splitting mid-thought loses meaning
- Missed retrieval — the right passage isn't found, so the answer is wrong
- Stale or conflicting documents — garbage in, confident garbage out
- Permission leaks — retrieving documents a user shouldn't see
- Prompt injection — malicious text inside documents steering the model3
Evaluate it
Measure retrieval (did we find the right passages?) separately from generation (did we answer correctly from them?). Build a test set of real questions with known answers and sources, and re-run it whenever you change chunking, embeddings, prompts or models. Anthropic's guidance for building AI systems stresses starting simple and measuring before adding complexity2.
RAG powers most "chat with your documents" features — see how to add AI to an existing app and how to build a chatbot for your business.
Key terms, defined
RAG — retrieval-augmented generation. Embedding — a numeric representation of meaning. Vector database — stores embeddings for similarity search. Chunk — a piece of a document used for retrieval. Hybrid search — combining keyword and meaning-based search. Citation — a reference to the source passage.
Frequently asked questions
What does RAG stand for?
Retrieval-augmented generation — combining a retrieval step (finding relevant information) with a generation step (an AI model writing an answer using that information).
Who invented RAG?
The term was introduced in a 2020 research paper by Patrick Lewis and colleagues, 'Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks'.
What's the difference between RAG and fine-tuning?
RAG gives the model relevant information at question time, so knowledge stays current and traceable. Fine-tuning changes the model's behaviour or style through training. Use RAG for knowledge; consider fine-tuning for consistent formats or specialised behaviour.
What is a vector database?
A database that stores embeddings — numeric representations of meaning — so it can find passages similar in meaning to a question, not just matching words.
Does RAG stop hallucinations?
It reduces them by grounding answers in sources, but doesn't eliminate them. Good retrieval, instructions to answer only from sources, citations and evaluation all help.
When should I use RAG?
When answers depend on information the model doesn't have or that changes — company documents, product catalogues, policies, support articles, recent data.
Sources
- Lewis et al. — Retrieval-augmented generation for knowledge-intensive NLP tasks (2020)
- Anthropic — Building effective agents
- OWASP — Top 10 for Large Language Model Applications
- Checked October 23, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Where to go from here
- How to add AI to an existing app — RAG in a real product
- How to build a chatbot for your business — a common RAG use case
- Career guidesRAG engineer skills employers want in 2027: chunking, embeddings, hybrid search, re-ranking, grounded prompts, citations and evaluationRAG skills AI engineering employers want in 2027: chunking, embeddings, hybrid search, re-ranking, grounding, citations, ingestion and RAG evaluation.
- Tech explainedHow does ChatGPT work? Tokens, transformers, training and why it sometimes makes things upHow ChatGPT works, in plain English: it predicts the next token using a transformer trained on vast text, then tuned with human feedback. With diagrams.
- AI & vibe codingWhat is MCP? The Model Context Protocol explained simply — and why it mattersWhat MCP (Model Context Protocol) is, how servers and clients work, why it became the standard for connecting AI to tools and data, and the security basics.
- AI & vibe codingWhat is an AI agent? A plain-English explanation, with examples you already useWhat an AI agent is, how it differs from a chatbot and a workflow, the loop that powers it, everyday examples and the risks.
- AI & vibe codingChatGPT vs Claude vs Gemini: which AI assistant is best for your work?ChatGPT vs Claude vs Gemini for real work — writing, analysis, coding, research, integrations and India plans — plus a test to choose.
- AI & vibe codingHow to add AI to an app: choosing features, architecture, evaluation, cost and trustHow to add AI features to an existing app: pick a user problem, choose the pattern, architecture, evaluation, cost control, privacy and UX.
Get the ideas, not the noise.
Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.
Make your AI feature something users trust.
A behavioral UX audit of your app or SaaS product: why users hesitate, stall and leave, and a prioritized fix list. Fixed price, 2–4 weeks.