Page 8 of 13 · Systems of agents
Multi-agent orchestration
More agents is not better. Each one adds cost, latency and a lossy handoff. Split work only when it divides cleanly, give every worker a precise brief, and put a check where a mistake would be expensive.
6 runnable samples1 prediction3 questions
Level 1 · The rules
The patterns
Nearly every multi-agent system is one of these, or two combined.
| Pattern | How it works | Good for | Watch out for |
|---|---|---|---|
| Prompt chaining | Fixed sequence, each output feeds the next | Predictable multi-step tasks | Errors compound down the chain |
| Routing | A classifier sends work to a specialist or model tier | Mixed requests, cost control | Silent misroutes unless logged |
| Orchestrator-worker | A lead splits the task, workers execute, the lead merges | Research, large refactors, audits | Worker quality equals brief quality |
| Fan-out / fan-in | Parallel workers, results combined | Independent subtasks | Token cost multiplies |
| Evaluator-optimizer | One produces, another critiques, repeat | Specs, code quality, writing | Same model may agree with itself |
| Verifier / skeptic | A final agent re-checks against different evidence | Research, data accuracy | Useless if it sees the same evidence |
| Human gate | Pause for approval at set points | Merges, deploys, irreversible steps | Too many gates and people rubber-stamp |
Level 1 · The rules
Fan-out with a budget
A research team of one data agent, four researchers and one skeptic can easily read hundreds of thousands of tokens. Know the math before you launch it.
workers = { "data": 60_000, "storms": 55_000, "smoke": 50_000, "rain_fog": 52_000, "trends": 58_000,}skeptic = 90_000total = sum(workers.values()) + skepticprice_per_million = 3.00print(f"{len(workers)} workers + 1 skeptic = {total:,} tokens")print(f"estimated input cost: ${total / 1_000_000 * price_per_million:.2f}")
[ OUTPUT ]
Press Run to see the output this code printed when the course was built.
class BudgetExceededError(Exception): passdef run_team(tasks, cap_usd, cost_per_task=0.40): spent = 0.0 for task in tasks: spent += cost_per_task if spent > cap_usd: raise BudgetExceededError( f"${spent:.2f} spent, cap ${cap_usd:.2f}, stopped at {task}" ) return spentrun_team([f"research-{i}" for i in range(12)], cap_usd=3.00)
[ TRACEBACK ]
Press Run to see the output this code printed when the course was built.
Level 2 · Why it works
Model tiering and depth limits
Strong models plan and review; cheaper models execute. And agents that can start agents need a depth limit, or a confused planner recurses.
TIERS = { "plan": "strong", "review": "strong", "implement": "mid", "format": "small", "search": "small",}PRICE = {"strong": 15.0, "mid": 3.0, "small": 1.0}tasks = ["plan", "search", "implement", "implement", "format", "review"]tiered = sum(PRICE[TIERS[t]] for t in tasks)all_strong = PRICE["strong"] * len(tasks)print( f"tiered: {tiered:.0f} units, all strong: {all_strong:.0f} units, saving" f" {1 - tiered / all_strong:.0%}")
[ OUTPUT ]
Press Run to see the output this code printed when the course was built.
def planner(task): subtask = "plan: " + task return planner(subtask)planner("build login")
[ TRACEBACK ]
Press Run to see the output this code printed when the course was built.
def delegate(task, depth=0, max_depth=2): if depth >= max_depth: return f"executed '{task}' at depth {depth}" return delegate("sub-" + task, depth + 1, max_depth)print(delegate("build login"))
[ OUTPUT ]
Press Run to see the output this code printed when the course was built.
Level 2 · Why it works
A verifier only helps with different evidence
A skeptic that reads the same sources as the researcher mostly agrees with it. Give it a second source, a test suite, or a different model.
claim = {"metric": "rainy days 2023", "value": 74}same_source = {"rainy days 2023": 74}second_source = {"rainy days 2023": 67}def verify(claim, source, tolerance=3): actual = source[claim["metric"]] return ( "confirmed" if abs(actual - claim["value"]) <= tolerance else f"disputed: source says {actual}" )print("skeptic with same source:", verify(claim, same_source))print("skeptic with NOAA-style second source:", verify(claim, second_source))
[ OUTPUT ]
Press Run to see the output this code printed when the course was built.
Level 2 · Watch it
Step through the mental models
Four short programs, one idea each: a loop turn filling the context, a hook blocking an edit, a brief losing a constraint, and compaction dropping a rule. Predict before each step.
[ PROGRAM ]
context = ["system prompt", "user: fix the failing test"]action = {"tool": "run_tests"}context.append("tool: 3 failed in cart.test.ts")action = {"tool": "edit", "path": "src/cart.ts"}context.append("tool: edited src/cart.ts")context.append("tool: 42 passed")answer = "Fixed rounding in cart total"
Context window
- system prompt
- instructions · stable
- cached
- user
- task · fix the failing test
- loaded
Harness state
steps 0 of 40 · $0.00
Step 1 of 7, line 1. The run starts with two things in context: the instructions and the task.
Predict: The model can't run anything. What does it produce next?
Level 3 · Mastery
Describe a pipeline in this vocabulary
A common shape for spec-driven delivery, described with the words from this page. Practice explaining your own system the same way:
Plan phase
Orchestrator-worker: planning agents turn a request into a spec and tasks. Strong models.Human gate
Review the spec before any code is written. The cheapest place to fix a mistake.Build phase
Handoff with a written brief per task; cheaper models implement, tests act as sensors.Verify and gate
An independent reviewer and the full test suite before merge.
Predict the output
Level 2 answer before you look
Check your understanding
Levels 1 to 3 open a question to see the answer
When should you not use multiple agents?
When the task needs tight shared context or does not split into independent parts. Each handoff loses information and adds cost.
What makes a verifier agent useful?
Independent evidence: a second source, tests, or a different model. Re-reading the same evidence mostly produces agreement.
What two limits does an agent team need?
A shared budget (tokens or dollars) and a delegation depth limit.