Agentic system design

This is an old revision. The current one is revision 2, 2026-04-15.

1. Start with one agent. Move to a flat supervisor with workers only when the numbers ask for it; a flat supervisor is enough up to about 10 agents.
2. Keep each agent's context separate and hand over short summaries (about 200 tokens), not whole transcripts. This cut tokens by 50 to 70%.
3. Prefer transfer over delegation when the worker can answer the user itself. It halved latency and cut one task from 10.2K to 7.2K tokens.
4. Lay out prompts for the cache. Static content first, then a checkpoint that moves down each turn, then the objective last. A cache hit rate under 90% is treated as an outage.
5. Compact the context at 70% full, by appending the summarize instruction instead of swapping the system prompt. That keeps the cache and was 80.7% cheaper ($0.3315 to $0.0639).
6. Run independent tool calls in parallel. It cut latency 3 to 5 times and cost 30 to 40%.

All revisions:r2, r1