AI glossary Models & Architecture
What is retrieval-augmented generation (RAG)?
RAG (Retrieval-Augmented Generation)
← All glossary termsA technique that retrieves relevant passages from a knowledge base and gives them to a language model as context, so its answer rests on those sources.
Explained
Retrieval-augmented generation (RAG) combines a search step with a language model. When a question arrives, the system looks up the most relevant passages in a document collection, usually through a vector index, and passes them to the model together with the question. The model then writes its answer from that material instead of relying only on what it learnt in training. This keeps answers tied to current, private sources, lets the system cite where each answer came from and reduces hallucinations, without retraining the model.
How RAG works
- Ingestion: documents are split into chunks and each chunk is turned into an embedding, a vector that captures its meaning.
- Indexing: the vectors are stored in a vector database or search engine, often alongside a keyword index.
- Retrieval: the question is embedded the same way and the closest chunks are fetched, sometimes reordered by a reranker.
- Generation: the model receives the question plus the retrieved chunks and writes the answer, ideally citing them.
RAG or fine-tuning?
RAG changes what the model can see at the moment it answers; fine-tuning changes the model itself. RAG suits knowledge that changes often, must stay under access control or needs to be cited. Fine-tuning suits a stable style, format or narrow task that the model should perform without long instructions.
The two are not exclusive. Many production systems use a general model, retrieval for the facts and, only where it pays off, a fine-tuned model for tone or structure.
Limits and what to watch
- Retrieval quality sets the ceiling: if the right passage is not retrieved, the model cannot use it.
- Chunking, metadata and permissions need care, or users see answers built from documents they should not access.
- The model can still misread or over-generalise the context, so answers need evaluation against real questions.
- Corpus-wide questions ("what are the main themes?") are hard for plain RAG and may need summarisation or graph-based approaches.
Frequently asked questions
What does RAG stand for in AI?
RAG stands for retrieval-augmented generation. The term comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University.
Does RAG eliminate hallucinations?
No. It reduces them by giving the model relevant sources, but the model can still ignore, misread or go beyond the retrieved text. Good prompts, citations and systematic evaluation are still needed.
Do I need a vector database for RAG?
Not always. Vector search is the most common retrieval method, but RAG can use keyword search, a hybrid of both, SQL queries or a knowledge graph. The right choice depends on the data and the questions people ask.
What is the difference between RAG and fine-tuning?
RAG adds information at query time by retrieving documents; fine-tuning adjusts the model's weights with extra training. RAG is easier to update and audit, while fine-tuning is better for teaching a consistent behaviour or format.
What is agentic RAG?
Agentic RAG lets an AI agent decide when and how to search: it can reformulate the query, search several sources, check what it found and search again. It handles complex questions better than a single retrieval step, at the cost of more latency and more calls to the model.
Decision guides that use this term
- RAG vs fine-tuning: which one does your enterprise use case need?
- RAG vs enterprise search: do your people need answers or documents?
- Fix the data first or start with AI now: which comes first?
- GraphRAG vs vector RAG: when does your knowledge assistant need a graph?
- Long context windows vs RAG: do you still need a retrieval pipeline if the model can read the whole dataset?
How to put it to work
- How to answer HR policy questions with AI
- How to automate email with AI
- How to automate employee onboarding with AI
- How to automate the IT service desk with AI
- How to build a company knowledge base with AI
- How to build an AI contact centre
- How to keep expert knowledge when people retire
- How to manage company documents with RAG
- How to monitor regulatory change with AI
- How to prepare sales meetings with AI
- How to review contracts with AI
Where it shows up at Thinkia
Further reading
Related terms
Sources
- Lewis et al. · Retrieval-augmented generation for knowledge-intensive NLP tasks (arXiv)
- Anthropic · Introducing contextual retrieval
- Gao et al. · Retrieval-augmented generation for large language models: a survey (arXiv)
Updated: 2026-10-21