Cost, latency and shipping

A working agent and a shippable agent are different things. The gap is usually cost, speed and the ability to turn it off.

On cost, three levers do most of the work. Use a smaller model for the parts that do not need the largest — routing, classification, extraction — and keep the capable model for judgement. Arrange your prompt so the stable prefix caches. And cut context: fewer tools, shorter results, compaction on long runs.

On latency, run independent tool calls concurrently, stream the answer so a person sees progress, and consider whether the work needs to happen while someone waits at all. A great deal of agent work is better as a background job that produces something for review.

Release in the order that keeps mistakes cheap. Let it observe and propose while a person does the work, and compare. Then let it act on reversible things with a person reviewing after. Then let it act unattended on the narrow class it has proven itself on, with the irreversible acts still requiring confirmation. Each stage produces the evidence needed to justify the next, and if the evidence does not arrive, you have learned something before it cost you a customer.

One last thing, which is the through-line of this whole course: the goal was never an agent. It was the work getting done reliably and affordably. If a workflow does it, ship the workflow. If a single well-designed tool removes the need for the loop entirely, ship that. The engineers who build the best agentic systems are the ones most willing not to.