Agentic system design
This is an old revision. The current one is revision 2, 2026-04-15.
- Start with one agent. Move to a flat supervisor with workers only when the numbers ask for it; a flat supervisor is enough up to about 10 agents. (entry 1)
- Keep each agent's context separate and hand over short summaries (about 200 tokens), not whole transcripts. This cut tokens by 50 to 70%. (entry 1)
- Prefer transfer over delegation when the worker can answer the user itself. It halved latency and cut one task from 10.2K to 7.2K tokens. (entry 1)
- Lay out prompts for the cache. Static content first, then a checkpoint that moves down each turn, then the objective last. A cache hit rate under 90% is treated as an outage. (entry 1)
- Compact the context at 70% full, by appending the summarize instruction instead of swapping the system prompt. That keeps the cache and was 80.7% cheaper ($0.3315 to $0.0639). (entry 1)
- Run independent tool calls in parallel. It cut latency 3 to 5 times and cost 30 to 40%. (entry 1)
All revisions:r2, r1