Context engineering: the job prompt engineering grew into

Context engineering is deciding, for every request, exactly what the model gets to see: the instructions, the knowledge retrieved for this question, what the system remembers about the user, the tools it can call, the results of tools it already called, and the conversation so far. Prompt engineering is the part of that about wording the instructions. In production, it is rarely the part that breaks.

01What is context engineering?

A model knows two things when it answers: what it learned in training, and what is in its context window for this request. You cannot change the first at request time. Context engineering is the discipline of controlling the second: choosing the sources, selecting from them, fitting the result into a budget, and putting it in an order and format the model uses well.

The term caught on as teams moved from single prompts to agents and retrieval systems, where most of the context is no longer written by a person. It is assembled by code, on every request, from data that changes.

02How is context engineering different from prompt engineering?

Good instructions still matter. They are simply one input among six or seven, and usually the most stable one.

03What goes into a model's context?

• Instructions: the system prompt, rules and output format.
• The request: what the user asked or the event that triggered the run.
• Retrieved knowledge: passages from your documents and data (see how RAG works).
• Memory: facts kept about the user or task from earlier sessions (see agent memory, planning and stopping).
• Tool definitions: the functions the model may call, with their schemas.
• Tool results and history: what earlier steps returned, and the conversation so far.
• Task state: where a multi-step process stands, such as which approvals are done.

Each source has its own owner, freshness and permission rules. Context engineering is where they meet.

04Why does more context make answers worse?

Because a model does not read its context the way a database runs a query. Everything in the window competes for attention. Long contexts dilute the passages that matter, and models are measurably worse at using information buried in the middle of a long input than at the start or end. Every extra token also adds cost and latency, and every stale or irrelevant passage is one more thing the model can confidently repeat.

A large context window is a ceiling, not a target (see a context window is not memory).

05How do you assemble context for each request?

Treat it as a build step with a budget, not as string concatenation:

BUDGET = {                       # tokens per section; the total stays well under the window
    "instructions": 1_500,
    "task_state": 500,
    "memory": 800,
    "knowledge": 4_000,
    "history": 1_500,
}

def assemble(req: Request, user: User) -> Context:
    ctx = Context(instructions=load_prompt(req.task, version=req.prompt_version))
    ctx.add("task_state", state_store.summary(req.run_id), budget=BUDGET["task_state"])
    ctx.add("memory", memory.relevant(user, req.text, scope=user.permissions), budget=BUDGET["memory"])
    passages = retrieve(req.text, scope=user.permissions)          # filtered before ranking
    ctx.add("knowledge", dedupe(passages), budget=BUDGET["knowledge"], cite=True)
    ctx.add("history", recent_turns(req.conversation), budget=BUDGET["history"])
    trace.record_context(req.id, ctx.manifest())                 # what went in, from where, how many tokens
    return ctx

Four habits carry most of the value:

1. Budget per section, so one source cannot crowd out the others.
2. Permission filtering before selection, so nothing reaches the model that the user could not see (see filter by permission before you rank).
3. Provenance on every passage, so the model can cite it and you can trace a bad answer to its source.
4. A manifest in the trace, so you can see what the model actually saw when an answer goes wrong (see trace and cost every model call).

06How do agents change the problem?

An agent calls the model in a loop, and the naive design appends every tool result and every thought to the context. By step ten the window is full of old search results, half of them irrelevant, and the model starts losing the thread.

Three changes fix most of it:

• Keep task state outside the model. A small, structured record of what is done and what is next is more reliable than a long transcript the model has to re-read.
• Summarise tool results as they arrive. Keep the facts the task needs; drop the raw payload, or store it and keep a reference.
• Rebuild the context each step from state, memory and fresh retrieval, instead of growing one transcript forever. This is sometimes called compaction.

The model can only be as good as what you put in front of it. Context engineering is choosing that, on purpose, every time.

07How do you know the context is right?

Measure it the same way as everything else, with an evaluation set (see ship a prompt change with an eval):

• Was the needed information in the context? Check this separately from whether the answer was right. If it was missing, the fix is retrieval or memory, not the prompt.
• Remove one section at a time and re-run the eval. Sections that do not change the score are cost without benefit.
• Watch tokens per request in your traces. A slow upward drift is usually context nobody decided to add.

08Where to go next

The vocabulary used here, from tokens to memory, is defined in the glossary. For the pieces of the context in depth, read how RAG works, a context window is not memory and agent memory, planning and stopping.