Skip to main content

AI glossary Data & Retrieval

What is context engineering in AI?

← All glossary terms
In short

The practice of deciding what information a model sees at each step (instructions, retrieved documents, tool results, memory) and fitting it into the context window.

Explained

Context engineering is the discipline of designing everything that goes into a model's context window, not only the prompt. It covers which documents are retrieved, which tool outputs are kept, how earlier conversation is summarised, what long-term memory is recalled and how all of it is ordered within a limited token budget. The idea is that a model can only reason over what it is given, so the quality of an answer depends as much on the context as on the model. It is the practical backbone of reliable RAG systems and agents, especially in long tasks where context accumulates quickly.

Context engineering vs prompt engineering

Prompt engineering focuses on how to word the instruction. Context engineering asks a broader question: of all the information available, what should the model see right now, in what form and in what order?

In a single question-and-answer exchange the two overlap. In an agent that runs for dozens of steps, calling tools and reading documents, the prompt is a small part of the context. The rest is retrieved content, tool results and history, and managing that is where most quality problems are won or lost.

Main techniques

  • Retrieval: bring in only the passages relevant to the current step, rather than whole documents.
  • Compaction: summarise older conversation or tool output so the essentials survive and the noise goes.
  • Structured memory: keep notes, decisions or task state outside the window and recall them when needed.
  • Tool design: return concise, well-structured results so tool output does not flood the context.
  • Isolation: give sub-agents their own clean context for a sub-task and pass back only the result.
  • Ordering and budgeting: place the most important material where the model uses it best and leave room for the answer.

Why it matters in production

Larger context windows do not solve the problem on their own. Models tend to use information less reliably as the context grows, irrelevant content distracts them, and every token adds cost and latency.

In practice, many failures attributed to the model are context failures: the right document was not retrieved, an outdated instruction stayed in the history, or a tool returned too much. Treating context as something you design, test and monitor makes agents and RAG systems more predictable.

Frequently asked questions

What is the difference between context engineering and prompt engineering?

Prompt engineering is about writing the instruction well. Context engineering covers everything the model receives: instructions, retrieved documents, tool results, history and memory, and how they are selected and fitted into the context window.

Why is context engineering important for AI agents?

Agents run many steps and accumulate tool outputs and history. Without active management, the context fills with stale or irrelevant material, quality drops and costs rise. Context engineering keeps each step focused on what matters.

Does a bigger context window remove the need for context engineering?

No. A larger window lets you include more, but models do not use every part of a long context equally well, and more tokens mean more cost and latency. Choosing what to include still matters.

Is RAG a form of context engineering?

Yes. Retrieval-augmented generation is one of its main techniques: it decides which external content enters the context. Context engineering also covers memory, summarisation, tool output and ordering.

Where it shows up at Thinkia

Products

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert