What shipped, and what the numbers said
By the end: Read an agent programme’s results honestly, including what it did not achieve.
Six weeks after the rebuild, the numbers had moved enough to be worth reporting — and one of them had not moved at all, which is the part worth reading closely.
- Sent without a human edit: 38% → 71%. Almost all of the gain came from the two workflow branches.
- Cost per ticket: $0.47 → $0.09. Fewer steps, a smaller model on classification, and a cacheable prompt prefix.
- Median time to first reply: 41 minutes → 6 minutes for the routed classes; unchanged for everything else.
- Wrong refunds issued: 2 → 0, by construction rather than by care.
- Billing disputes resolved without a person: 22% → 26%. Four points, on the branch that kept the agent.
That last line is the honest one. The agentic branch — the interesting part, the part that justified the project in the founder’s mind — moved four points. The boring routed workflows delivered nearly all the value. The team had been most excited about the least productive third of the system.
What they would tell you they would do differently: read 300 tickets before writing any code rather than after, ship the observing-and-proposing stage first instead of going straight to drafting, and put the cost-per-ticket chart up on day one rather than after it had cost them $80 to notice a broken carrier.
And one thing they got right by accident. Because refunds always required a person, the injection in week three was a curiosity rather than an incident. Nobody had anticipated that specific attack. They had simply decided, early and for unrelated reasons, that money leaving the company needed a human. That decision paid for itself the first time it was tested — which is the usual way a permission boundary earns its keep.
End of module 8
Case Study: Meridian Tools
That was the last module. The final exam and practice quizzes are on the course page. Go to the course page →

