The context window is a budget
By the end: Account for what fills a context window during an agent run and explain why quality falls before the limit does.
Every request carries the system prompt, the tool declarations, the whole conversation so far, and the new turn. That total is the context, and it has a hard ceiling. What surprises people is that hitting the ceiling is rarely the real problem — quality degrades well before it, because relevant facts get diluted among irrelevant ones.
A useful mental image is a desk. A model works with everything on the desk at once. Adding a document nobody needs does not just cost desk space; it makes the useful documents harder to find. Long, rambling context produces agents that miss instructions they were given ten turns ago.
- System prompt and rules: 900 tokens, sent every turn.
- Tool declarations, nine tools: 1,400 tokens, sent every turn.
- Seven previous turns with their results: 11,000 tokens and growing.
- The new tool result: 600 tokens.
- The share of that spent on the current decision: small, and shrinking every step.
One structural trick pays for itself immediately. Keep the stable part — system prompt, tools, unchanging instructions — at the front and genuinely unchanging, and let volatile content accumulate after it. Providers can cache that stable prefix, so you pay full price for it once rather than on every turn. Reordering your prompt so the fixed part is truly fixed is often the cheapest cost saving available.