State between turns
By the end: Decide what belongs in the conversation, what belongs in your own storage, and why.
Because the model is stateless, there are exactly two places state can live: inside the conversation you resend each turn, or outside it in your own storage. Choosing correctly keeps agents cheap and debuggable.
In the conversation belongs what the model needs to reason with right now: the request, what it has tried, what those attempts returned. Outside belongs everything else — the durable record of what happened, the artefacts produced, anything another part of your system needs, and anything too large to re-send.
- In the conversation: the ticket text, the order status it fetched, the doc paragraph it found.
- In your storage: the run id, every tool call and its result, tokens and cost, the final decision.
- Neither: the entire 40-page shipping policy. Retrieve the relevant paragraph instead.
A useful discipline: write your own record of the run as it happens, not afterwards from the conversation. The conversation is a working surface that you may compact or truncate; your record is the thing you will read at three in the morning when a customer asks why the agent said what it said. Module 6 returns to this as an accountability requirement rather than a debugging convenience.
End of module 2