AI agent orchestration: patterns, platforms and when multi-agent is worth it

One AI agent with a handful of tools can do a lot. The trouble starts when you add a second, third or tenth agent: who decides which agent works next, what each one is allowed to see, and how do you stop them from repeating each other's work or running up a large bill?

That is the job of AI agent orchestration. This guide explains the core patterns in plain terms, shows what multi-agent systems actually cost according to the teams that run them, compares the main orchestration platforms as of November 2026, and gives you a tested Python orchestrator, standard library only, that runs subagents in parallel under a hard budget.

What is AI agent orchestration?

IBM defines AI agent orchestration (opens in a new tab) as "the process of coordinating multiple specialized AI agents within a unified system" to reach shared objectives. GitHub's explainer (opens in a new tab) puts it almost the same way and adds the useful part: orchestration is a control layer for execution, context and collaboration across agents, and it is where guardrails such as cost limits and human approval checkpoints are enforced.

In practice an orchestrator answers four questions on every run:

1. Who works on this? Which agent, or agents, get each subtask.
2. In what order? One after another, all at once, or decided on the fly.
3. With what context? What each agent is told, and what it is not shown.
4. When is it done? The stopping condition, the budget and what happens when something fails.

The orchestrator itself can be another model (a lead agent that plans and delegates) or ordinary code that runs agents in a fixed shape. Most real systems mix both. If you need a refresher on what sits inside a single agent, our guide to agentic AI architecture covers the seven components. This article is about what happens when there is more than one.

Agent orchestration vs related terms

GitHub's article draws the same line on MCP: it lists the protocol separately from orchestration, as a standard way to give agents consistent context and access to apps and data.

Two ways to orchestrate: let the model decide, or write the flow

The OpenAI Agents SDK documentation (opens in a new tab) describes two broad approaches, and the distinction applies to every framework:

• Orchestrating via the LLM. A lead agent plans the task and decides which agents or tools to call. This is flexible and handles open-ended work, but each run can take a different path.
• Orchestrating via code. Your code determines the flow: chain agents so one's output becomes the next one's input, use structured outputs your code can inspect, pick the next agent based on a category, loop until an evaluator agent says the output passes, or run agents in parallel with Python primitives like asyncio.gather.

The docs are direct about the trade-off: orchestrating via code "makes tasks more deterministic and predictable, in terms of speed, cost and performance." A good rule is to let code own everything you can predict and let a model decide only the parts you cannot.

Agents as tools vs handoffs

Inside LLM-driven orchestration there is a second choice that shapes the whole system. With agents as tools, a manager agent keeps control of the conversation and calls specialists the way it would call any tool; the specialist does its subtask and reports back. With a handoff, the specialist "becomes the active agent" and takes over the conversation. LangChain's multi-agent docs (opens in a new tab) use the same split, calling the first pattern subagents.

Use agents as tools when one agent should own the user relationship and the final answer, such as a research lead that combines findings. Use handoffs when the right specialist should talk to the user directly, such as a support triage agent passing a billing question to a billing agent.

The main multi-agent orchestration patterns

Frameworks name these patterns slightly differently, but they describe the same handful of shapes. Microsoft Agent Framework (opens in a new tab) ships five as built-in orchestrations; the table adds orchestrator-workers and routing, which LangChain and Anthropic document.

IBM groups the same ideas by control model: centralized (one orchestrator directs everything), decentralized (agents coordinate directly), hierarchical (orchestrators in tiers) and federated (independent agents or organizations collaborate without fully sharing data, useful in healthcare and banking). Databricks' guide (opens in a new tab) notes that centralized orchestration offers auditability but creates a single point of failure, and recommends starting with a centralized approach on one high-impact workflow.

What multi-agent orchestration actually costs

Most articles on this topic list the benefits of multi-agent systems and stop. The teams that publish real numbers paint a more balanced picture.

Anthropic's account of how it built its multi-agent research system (opens in a new tab) is the most detailed public example. Its lead agent, running Claude Opus 4, coordinated Claude Sonnet 4 subagents and outperformed a single Opus 4 agent by 90.2% on Anthropic's internal research evaluation. The price: agents use about 4 times the tokens of a chat, and multi-agent systems "use about 15× more tokens than chats." Anthropic concludes that multi-agent systems need tasks where the value is high enough to pay for that performance.

Independent research adds a warning. In Why Do Multi-Agent LLM Systems Fail? (opens in a new tab), a team of researchers annotated more than 1,600 traces from seven popular multi-agent frameworks. Their abstract opens with the finding that performance gains on popular benchmarks "are often minimal." They identified 14 failure modes in three groups:

• System design issues: agents disobey the task specification, repeat steps, lose conversation history or do not know when to stop.
• Inter-agent misalignment: agents derail from the task, withhold information from each other, fail to ask for clarification, or reason one way and act another.
• Task verification: agents end too early, or check their work incompletely or incorrectly.

The authors argue these failures come largely from system design, not just model limits, so a better model alone will not fix them. That is good news in one sense: most of these problems are orchestration problems you can design out.

LangChain's documentation makes the cost concrete per pattern. In its worked comparison, a one-shot request took 4 model calls with subagents and 3 with handoffs, skills or a router. A multi-domain request took about 9,000 tokens with subagents or a router, versus 14,000 or more with handoffs. The exact figures depend on the task, but the lesson holds: the pattern you choose changes your bill.

When multi-agent is worth it vs a single agent

Every major vendor's guidance points the same way: start with one agent. GitHub's article says a single agent "can execute certain tasks reliably without introducing unnecessary complexity." LangChain says multi-agent patterns are "particularly valuable when a single agent has too many tools." Anthropic notes that domains requiring all agents to share the same context are a poor fit today, and that most coding tasks have fewer truly parallel parts than research.

Work through these questions in order:

1. Is the task a single, predictable step? Use one model call or a plain workflow. No orchestration needed.
2. Does one agent cope with its tools and instructions? If it picks the right tool and the prompt is maintainable, keep one agent and improve its tools.
3. Does the work split into independent parts? If yes, orchestrator-workers or concurrent agents can save time by running in parallel.
4. Do parts need different permissions or owners? Separate agents with narrow access are easier to secure and audit than one agent that can do everything.
5. Must every agent see the same evolving context? If yes, stay with one agent. Splitting tightly coupled work causes the misalignment failures described above.
6. Is the result worth several times the cost? If not, the single agent wins even if it is slightly worse.

If you are still deciding whether you need agents at all, our practical guide to AI agents for business covers where they pay off.

An orchestrator-workers example in Python (tested)

The script below is a minimal orchestrator-workers system for a realistic job: preparing a sales rep for a client meeting. A planner returns subtasks as structured JSON, three workers run in parallel, and a synthesizer combines their notes. It also enforces a hard cap on model calls. To keep it runnable without an API key, fake_llm() stands in for a real model. We ran it with Python 3.13 on November 3, 2026, using only the standard library.

# Orchestrator-workers with parallel subagents, scoped briefs and a budget.
# fake_llm() stands in for a model so the script runs offline.
# Tested with Python 3, standard library only.
import asyncio, json, time

BUDGET = {"calls": 0, "max_calls": 8}  # hard cap on model calls

async def fake_llm(role, prompt):
    BUDGET["calls"] += 1
    if BUDGET["calls"] > BUDGET["max_calls"]:
        raise RuntimeError("budget exceeded: stopping run")
    await asyncio.sleep(0.1)  # pretend network latency
    if role == "planner":
        return json.dumps([
            {"agent": "crm", "objective": "Summarize account history for Acme",
             "output": "3 bullets", "limits": "CRM data only"},
            {"agent": "support", "objective": "List open tickets for Acme",
             "output": "ticket ids + one-line status", "limits": "last 90 days"},
            {"agent": "news", "objective": "Find Acme news from this month",
             "output": "2 headlines with dates", "limits": "public sources only"},
        ])
    if role == "synthesizer":
        return "BRIEF: " + " | ".join(line.split(": ", 1)[1] for line in prompt.splitlines())
    return f"{role} done ({prompt.split(';')[0]})"

async def worker(task):
    # Each worker sees only its own brief, not the whole conversation.
    brief = f"{task['objective']}; format={task['output']}; limits={task['limits']}"
    return task["agent"], await fake_llm(task["agent"], brief)

async def orchestrate(goal):
    plan = json.loads(await fake_llm("planner", goal))        # 1. plan
    print(f"plan: {len(plan)} subtasks")
    results = await asyncio.gather(*(worker(t) for t in plan))  # 2. fan out
    for agent, out in results:
        print(f"  [{agent}] {out}")
    notes = "\n".join(f"{a}: {o}" for a, o in results)
    return await fake_llm("synthesizer", notes)               # 3. combine

t0 = time.perf_counter()
print(asyncio.run(orchestrate("Prep me for the Acme meeting")))
print(f"model calls: {BUDGET['calls']}, wall time: {time.perf_counter() - t0:.1f}s")

BUDGET.update(calls=0, max_calls=3)  # now a budget too small for the plan
try:
    asyncio.run(orchestrate("Prep me for the Acme meeting"))
except RuntimeError as err:
    print("second run:", err)

Running python3 orchestrator.py printed:

plan: 3 subtasks
  [crm] crm done (Summarize account history for Acme)
  [support] support done (List open tickets for Acme)
  [news] news done (Find Acme news from this month)
BRIEF: crm done (Summarize account history for Acme) | support done (List open tickets for Acme) | news done (Find Acme news from this month)
model calls: 5, wall time: 0.3s
plan: 3 subtasks
second run: budget exceeded: stopping run

Four details in this small script are the ones that matter in production:

• Every brief has an objective, an output format and limits. Anthropic found that without detailed task descriptions, subagents duplicated each other's work. Clear boundaries are the fix.
• Workers see only their own brief. Passing each worker the whole conversation wastes tokens and invites confusion. LangChain calls this context engineering and puts it "at the center of multi-agent design."
• Parallel workers cut wall time. Each fake call takes 0.1 seconds, so five calls in sequence would take about 0.5 seconds. Running the three workers together brought the run to 0.3 seconds. Anthropic reports that parallel subagents and tool calls cut research time by up to 90% for complex queries.
• The budget is enforced by code, not by the model. The second run stops cleanly when the cap is hit. A model will not reliably police its own spending.

To make it real, replace fake_llm() with calls to your model provider and give each worker its own tools. For a walkthrough of a single agent with real tool calling, see how to build an AI agent.

AI agent orchestration platforms compared

You can write orchestration yourself, as above, or use a framework or platform that provides patterns, state, guardrails and tracing. Here is how the main options position themselves, based on their own documentation and GitHub's overview, as of November 2026.

Whatever you pick, GitHub's list of orchestration components doubles as a buying checklist for a multi agent orchestration platform:

• State store: tracks completed work so a run can resume after a failure instead of starting over.
• Policy engine and guardrails: rules as code and hard limits on risky actions.
• Human approval: a way to pause a run for review before an irreversible action.
• Observability: logs, metrics and a trace of every model call, tool call and handoff.
• Cost controls: budgets and usage caps per run and per agent.

Databricks adds checkpoints, retries, least-privilege access and immutable audit trails to that list. If you want hosted agents rather than a framework, our roundup of the best AI agents compares ready-made platforms.

Rules for orchestrating agents in production

1. Start centralized. One orchestrator, one workflow, clear logs. Move to handoffs or peer patterns only when you have a reason.
2. Write briefs like task tickets. Objective, output format, tools to use and boundaries for every subagent.
3. Scale effort to the task. Anthropic's lead agent uses one agent for simple fact-finding and more than 10 subagents only for complex research. Encode that in your planner's instructions.
4. Cap everything in code. Maximum steps, maximum model calls, maximum spend and a timeout per agent.
5. Give each agent the narrowest permissions it needs. GitHub warns that overly broad permissions can lead to deleted repositories or exposed secrets.
6. Verify before you finish. Ending too early and weak checking form a whole failure category in the study above. Add an explicit verification step.
7. Trace every hop. When a multi-agent run goes wrong, the trace is the only way to see which agent caused it.

The bottom line

AI agent orchestration is the control layer that decides who works, in what order, with what context and under what limits. Prefer code for anything predictable, use a lead agent with subagents when work splits into parallel parts, and remember that multi-agent systems cost many times more than a single agent. Start with one agent, add more only when the evidence says you need them, and browse our AI guides for more on agents, models and tools.