Agentic system design

1. Start with one agent. Move to a flat supervisor with workers only when the numbers ask for it; a flat supervisor is enough up to about 10 agents.
2. Keep each agent's context separate and hand over short summaries (about 200 tokens), not whole transcripts. This cut tokens by 50 to 70%.
3. Prefer transfer over delegation when the worker can answer the user itself. It halved latency and cut one task from 10.2K to 7.2K tokens.
4. Lay out prompts for the cache. Static content first, then a checkpoint that moves down each turn, then the objective last. A cache hit rate under 90% is treated as an outage.
5. Compact the context at 70% full, by appending the summarize instruction instead of swapping the system prompt. That keeps the cache and was 80.7% cheaper ($0.3315 to $0.0639).
6. Start each tool as soon as its call has finished streaming, not when the whole message ends. About 50% faster on the median production trace, 1.20 to 1.50 times faster than plain parallel calls in the benchmark.
7. Do not start a tool early if it is not idempotent, takes under 50 ms, or needs another tool's result.

All revisions:r2, r1