Skip to main content

AI glossary Data & Retrieval

What is retrieval in RAG and search systems?

← All glossary terms
In short

The search step that fetches the documents or passages most relevant to a query, before they are shown or passed to a model.

Explained

Retrieval is the step in RAG and in search systems that finds, among a large collection, the documents or text passages most relevant to a query. It usually returns a small ranked list, the top-k results, from an index built in advance. It can work by matching keywords (sparse retrieval), by comparing embeddings in a vector space (dense retrieval) or by combining both. In a RAG system, what retrieval returns is all the model sees of your knowledge, so its quality sets the ceiling for the quality of the answer.

Sparse, dense and hybrid

  • Sparse retrieval matches the words in the query with the words in the documents, weighted by how distinctive they are. BM25 is the classic method. It is fast, transparent and good with exact terms such as codes and names.
  • Dense retrieval turns queries and passages into embeddings and finds the nearest ones in a vector index. It finds passages that mean the same thing even when they use different words.
  • Hybrid retrieval runs both and merges the results. In enterprise content, where exact terms and paraphrases both matter, it is often the safer default.

Reranking and filtering

First-stage retrieval is built for speed over large collections, so it returns a generous set of candidates. A reranker, usually a model that reads the query and each candidate together, then orders them more precisely and keeps the best few.

Filters are applied around this: by date, document type, language and, above all, by the user's permissions, so that retrieval never returns content the user is not allowed to see.

How to tell if retrieval works

  • Build a test set of real questions with the passages that answer them.
  • Measure recall at k: how often the right passage is among the first k results.
  • Measure ranking quality, for example with mean reciprocal rank.
  • Evaluate retrieval separately from generation, so you know which one to fix.
  • Review failures by hand: they often point to bad chunking, missing documents or vocabulary gaps.

Frequently asked questions

What does retrieval mean in RAG?

It is the R in retrieval-augmented generation: the step that searches your documents for passages relevant to the question. Those passages are then given to the language model as context for its answer.

What is BM25?

BM25 is a ranking function for keyword search. It scores documents by how often the query terms appear in them, adjusted for how rare each term is across the collection and for document length. It remains a strong baseline and a common part of hybrid search.

What is top-k in retrieval?

Top-k is the number of results the retriever returns, for example the five or ten most relevant passages. A higher k raises the chance of including the right passage but adds noise and cost when the passages are sent to a model.

Is dense retrieval always better than keyword search?

No. Dense retrieval is better at matching meaning, but keyword search is often better for exact identifiers, rare terms and names. That is why many systems combine both and rerank the merged results.

What is a reranker?

A reranker is a model that takes the query and a set of candidate passages and orders them by relevance more precisely than the first retrieval stage. It is slower per passage, so it is applied only to the shortlist.

Decision guides that use this term

How to put it to work

Where it shows up at Thinkia

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert