Context engineering

This block is about what sits in the model's window on each step: what fills it, four ways to keep it useful, and how compressing history can lose a rule the agent had to follow.

A Northwind customer disputes damage on order 4417. The agent keeps pulling the full order history and shipment documents. By step 15 it has 96,000 tokens in front of it, and the customer's actual complaint is a small part of that.

One whiteboard by the phone

Your colleague on the phone has one whiteboard. Anything they use for their next sentence has to be on it, and whatever is off the board does not exist for them. Between calls you wipe it and write it again, so you choose what goes up each time.

The whiteboard is the context window: all the tokens the model sees on one call. Context engineering is deciding what goes into it on each step. Anthropic's phrase for the goal is the smallest set of high-signal tokens that gets the result you want.

Windows have grown, to 1 million tokens on Claude Sonnet 5.5. Size still matters for two reasons. The previous block showed the first: each token is paid for again on every later call. The second is quality. Anthropic reports that a model's recall from its context falls as the context grows, and LangGraph's docs warn that most models do worse over long contexts, distracted by stale or off-topic content.

Here is step 15 of that run:

Tool results come first here. Tool schemas can be the next big item, and they are on the board before any work starts. Anthropic gives an example of five connected MCP servers whose 58 tools take about 55,000 tokens.

Four moves

1. Load only what the step needs. Your tool returns a short result plus a way to get more. The agent keeps the pointer and fetches details if it needs them. Anthropic calls this "just in time" retrieval.

{
  "order_id": "4417",
  "lines_total": 212,
  "lines_shown": 5,
  "next_cursor": "c_88f2",
  "lines": [
    {
      "sku": "LMP-204",
      "qty": 1,
      "status": "delivered_damaged"
    }
  ]
}

The same idea applies to schemas. On Anthropic's API, tools marked defer_loading: true stay out of the window until a tool search finds them, and the docs say this typically cuts definition tokens by over 85 percent. The price of loading on demand is more calls, each slower than having the data ready.

2. Compress old history. There are two strengths.

Clearing removes the content of old tool results and leaves a placeholder. The agent already used them. On Anthropic's API this is a beta feature, and your code sets it on the request:

{
  "context_management": {
    "edits": [
      {
        "type": "clear_tool_uses_20250919",
        "trigger": {
          "type": "input_tokens",
          "value": 60000
        },
        "keep": {
          "type": "tool_uses",
          "value": 3
        },
        "clear_at_least": {
          "type": "input_tokens",
          "value": 20000
        },
        "exclude_tools": [
          "get_refund_policy"
        ]
      }
    ]
  }
}

Compaction replaces older turns with a summary. Anthropic offers it in beta, either on request or automatically at a token threshold, which defaults to 150,000 input tokens. OpenAI has its own compaction too.

In this model, with a 12,000-token start and 8,000-token summaries, 60 calls send 4.85 million input tokens with compaction and 11.34 million without. That count already includes the two extra calls that write the summaries, because producing a summary is itself a billed model call.

Compression also changes the prefix, so the next call misses the cache from that point. Anthropic's advice is to clear enough at once to make that worthwhile, which is what clear_at_least is for, and to put a cache breakpoint at the end of the system prompt so it stays cached across compactions.

3. Keep notes outside the window. The agent writes what it has established to a file or a store, and reads it back when needed: "photo verified, carrier claim C-118 open." The next block covers storage that outlives the run.

4. Hand a subtask to a subagent. A separate agent with a clean window reads the 200 lines of supplier documents and returns a short answer. Anthropic describes subagents that may use tens of thousands of tokens and return a summary, often 1,000 to 2,000. The main agent never sees the rest, so it also cannot check it.

What compaction can lose

Whiteboards get wiped by someone in a hurry. Imagine a supervisor joined at step 12 and wrote: "Do not refund order 4417 until the address check is done." That sentence lives in the history. A summary written at step 24 keeps the gist of the case and drops the sentence. At step 25 the model sees "damage confirmed, refund due" and calls issue_refund.

Anthropic warns about this: overly aggressive compaction can lose subtle context whose importance only shows later. Its compaction docs also say that text instructions in mid-conversation system messages inside the summarised range stop applying once the summary replaces them.

Three things protect you, strongest first:

1. A rule that must hold is checked in your code before the tool runs. The earlier block on code decisions covers that gate.
2. A standing rule goes in the top-level system prompt, which stays outside the summary. A rule that appears mid-run gets saved by your code and added back to the prompt.
3. You tell the summariser what it must keep. Anthropic's instructions field replaces the default summary prompt entirely, so it has to say everything that matters. You can also keep recent turns word for word.

If you write compaction yourself, respect the message format as well. On Anthropic's API, tool results have to come immediately after the tool_use that asked for them, so a cut that separates the pair produces a request the API rejects.

Conclusion

• Context engineering decides what the model sees on each step. In tool-heavy runs, tool output and tool schemas often take the most space.
• The moves are loading on demand, compressing history, keeping notes outside and using subagents. Each has a cost.
• Compression can drop rules. Keep the ones that must hold outside the history, and enforce them in code.

Go deeper