Multi-agent orchestration patterns: orchestrator-worker, pipelines, handoffs and debate

Key points

• One agent with good tools is the default. Anthropic’s guidance is to find the simplest solution and add complexity only when needed.
• Pipelines and routers are workflows: your code fixes the path. Orchestrators, handoffs and debate let the model choose it.
• Orchestrator-workers fits tasks whose subtasks you cannot predict; each worker gets a precise brief and returns a compact result.
• Cost and latency multiply: Anthropic reports its multi-agent research system used about 15 times the tokens of a chat.
• Cemri et al. sort multi-agent failures into system design, inter-agent misalignment and task verification. Design and verification are yours to fix.
• Trace every agent as a span in one tree; without it, debugging a multi-agent run is guesswork.

When one agent is enough

An agent here is a model in a loop with tools, as in how LLM tool calling works. A multi-agent system runs several such loops, each with its own instructions, tools and context, and passes work between them. That can help, but it is rarely where to start.

Anthropic’s "Building effective agents" draws the line most teams now use. Workflows are systems where models and tools follow predefined code paths; agents are systems where the model directs its own process and tool use. Its advice is to find the simplest solution possible and increase complexity only when needed, and it notes that for many applications optimizing a single model call, with retrieval and in-context examples, is usually enough. It notes that agentic systems often trade latency and cost for better task performance, and that their autonomy means higher costs and the potential for compounding errors.

• Try one call first, then one agent with tools, then a fixed workflow, and only then several agents.
• Add a second agent when you can name the problem it solves: a context window that overflows, subtasks that can run in parallel, a step that needs different tools or permissions, or a draft that needs an independent check.
• Anthropic’s write-up of its multi-agent research system says domains where all agents must share the same context, or with many dependencies between agents, are not a good fit today, and that most coding tasks have fewer parallelizable parts than research.

The business-level trade-off is in AI workflows vs agents and multi-agent systems. This post is about the mechanics.

The patterns at a glance

The first six come from Anthropic’s guide, which treats routing, chaining, parallelization, orchestrator-workers and evaluator-optimizer as workflow building blocks. The OpenAI Agents SDK docs make the same split from the other side: orchestrating via the model, where it plans and picks the steps, and orchestrating via code, which is more predictable in speed, cost and performance. Real systems mix them: a router in front, a pipeline inside one branch, an evaluator at the end.

Pipelines and routing: your code holds the map

A pipeline is a sequence of model calls, each with a narrow job: extract, then outline, then draft, then check. Between steps, put programmatic gates, such as a schema check, a length check or a test run, so a bad intermediate result stops the chain instead of polluting everything after it. Pipelines trade latency for accuracy: each call is easier, but they run one after another.

A router is one cheap classification call followed by a specialized one. It lets you keep each specialist’s prompt short and focused, send easy inputs to a small model and hard ones to a large one, and give each branch only the tools it needs.

const ROUTES = {
  billing: { model: 'small', system: 'You answer billing questions. You cannot issue refunds.' },
  technical: { model: 'large', system: 'You debug technical problems step by step.' },
  general: { model: 'small', system: 'You answer general questions about the product.' },
} as const;
type Route = keyof typeof ROUTES;

async function handle(message: string): Promise<string> {
  const label = (await callModel(
    'Classify the message as billing, technical or general. Reply with the label only.',
    message, { model: 'small' },
  )).trim().toLowerCase();
  const key: Route = label in ROUTES ? (label as Route) : 'general';   // unknown label: safe default
  const route = ROUTES[key];
  return callModel(route.system, message, { model: route.model });
}

• Validate the label against a fixed set and fall back to a safe default. A model asked for one word will sometimes return a sentence.
• Log the label with the request, so you can measure routing accuracy on real traffic and retrain or reprompt the router.
• A router can be a classifier, an embedding nearest-neighbor lookup or plain rules. It does not have to be a large model.

Orchestrator-workers

In orchestrator-workers, a lead model breaks the task down at run time, sends subtasks to workers, reads their results and decides whether to send more or to answer. Anthropic’s guide recommends it for complex tasks where you cannot predict the subtasks needed, such as code changes across many files or research across many sources. The difference from parallelization is that the lead chooses the subtasks instead of your code.

type Subtask = { id: string; brief: string };
type Plan = { done: boolean; answer: string; subtasks: Subtask[] };

async function orchestrate(goal: string, maxRounds = 3, maxWorkers = 4): Promise<string> {
  const findings: string[] = [];
  for (let round = 0; round < maxRounds; round++) {
    const plan: Plan = JSON.parse(await callModel(          // validate with a schema in real code
      'You are the lead. Return JSON: {"done", "answer", "subtasks": [{"id", "brief"}]}. ' +
        'Each brief states the objective, output format, sources to use and what is out of scope.',
      `Goal: ${goal}\nFindings so far:\n${findings.join('\n') || '(none yet)'}`,
    ));
    if (plan.done) return plan.answer;

    const batch = plan.subtasks.slice(0, maxWorkers);       // cap the fan-out
    const results = await Promise.allSettled(
      batch.map((t) => callModel('You are a worker. Do only this task. Cite what you used.', t.brief)),
    );
    results.forEach((r, i) => findings.push(r.status === 'fulfilled'
      ? `${batch[i].id}: ${r.value}`
      : `${batch[i].id}: FAILED (${String(r.reason)})`));    // the lead sees failures too
  }
  return callModel('Write the final answer from these findings only. Say what is missing.', findings.join('\n'));
}

callModel(system, input, opts) stands in for any provider’s API. In a real system each worker is itself a tool-using agent, as in the agent loop, and the plan is parsed with a schema; see LLM structured outputs. The structure is what matters: bounded rounds, a capped fan-out, failures reported back to the lead, and a final synthesis step that is told to admit gaps.

Anthropic’s account of its research system is the most detailed public write-up of this pattern. It reports that vague instructions led subagents to misread tasks and duplicate each other’s searches, and that each subagent needs an objective, an output format, guidance on tools and sources, and clear task boundaries. It scaled effort to the question: one agent with a few tool calls for a simple fact, more subagents for comparisons and broad research. It also describes the cost of synchronous execution: the lead waits for each batch, so the whole system can be blocked by a single slow subagent.

Voting, evaluator loops and debate

These patterns use more model calls on the same problem to get a better answer, not to cover more ground.

• Voting runs the same prompt several times, or several different prompts, and aggregates: a majority answer, a threshold ("flag if two of three reviewers flag"), or the union of issues found. It helps most when the runs make different mistakes. Runs of one model on one prompt often fail the same way, so vary prompts or models.
• Evaluator-optimizer pairs a generator with an evaluator that returns specific feedback, and loops until the evaluator passes the draft. Anthropic’s guide says it fits when there are clear evaluation criteria and refinement provides measurable value. Give the loop a maximum number of rounds, and prefer checks code can run, such as tests, schemas or a linter, over a model’s opinion where you can.
• Debate, from Du et al. (2023), has several model instances propose answers, read each other’s answers and reasoning, and revise over several rounds until they converge on a common answer. The authors report that it improves mathematical and strategic reasoning across a number of tasks and the factual validity of generated content, reducing fallacious answers and hallucinations, and that it works with black-box models using the same procedure and prompts across tasks.

Debate makes one call per agent per round, and every round rereads everyone else’s answers, so the token count grows fast. Use it where a wrong answer is expensive and the question has a checkable answer. Agreement is not truth: agents that share a model can share a misconception and converge on it confidently.

Handoffs and hierarchies

A handoff passes control of the conversation from one agent to another. In the OpenAI Agents SDK, handoffs are represented to the model as tools: an agent named Refund Agent becomes a tool called transfer_to_refund_agent, and when the model calls it, the specialist becomes the active agent for the rest of the turn and answers the user directly. By default the new agent sees the conversation so far. The handoff() helper adds an input_type for metadata the model fills in, such as a reason, an on_handoff callback and an input_filter that changes what the next agent receives.

The SDK docs draw this line too: agents as tools, through Agent.as_tool(), suit cases where one agent should own the final answer. Handoffs have their own failure: two agents handing the user back and forth. Limit the number of handoffs per turn, and make each agent’s description say what it does not handle.

A hierarchy stacks orchestrators: a top lead delegates to team leads, who delegate to workers. It mirrors org charts and can split very large tasks, but each level summarizes what it heard, adds a round trip and is one more place where a detail can be lost. Use it only when a flat orchestrator’s context or fan-out is the measured bottleneck.

Shared state, message passing and context isolation

Agents coordinate in one of two ways, and most systems use both.

• Message passing: agents exchange text or structured messages, such as a brief down and a result up. It is explicit and easy to trace, and each agent sees only what it was sent.
• Shared state: agents read and write a common store, such as a plan, a task board, a file system or a database. It avoids copying large results through prompts, but needs rules for who may write what, and conflicts look like any other concurrent-write bug.

Context isolation is the main technical reason to split work across agents. Each subagent starts with a clean window, spends its tokens exploring and returns a compact result, so the lead’s context holds conclusions instead of raw search results. Anthropic describes subagents working in parallel with their own context windows as a form of compression, and says its subagents store their work in external systems and pass lightweight references back to the coordinator, which avoids a game of telephone. The lead also saved its plan to memory, because a context window over 200,000 tokens would be truncated; see AI agent memory.

Isolation cuts both ways. A worker does not know what the user said three turns ago unless the brief tells it, and a lead that receives a summary cannot see what the worker left out. Cemri et al. list loss of conversation history, information withholding and ignoring another agent’s input among the failure modes they observed.

Cost, latency, failure and observability

Every pattern multiplies model calls. An orchestrator with R rounds and W workers makes about R × (1 + W) + 1 calls before the workers’ own tool loops, and each call resends its context. Anthropic reports that in its data agents used about 4 times more tokens than chat interactions and multi-agent systems about 15 times more, and that token usage alone explained 80% of the variance in performance on the BrowseComp evaluation. More agents can buy quality, at a price measured in tokens. Latency follows the critical path: parallel workers cost the slowest worker’s time, sequential steps cost the sum. Levers for the bill are in how to reduce LLM costs.

"Why Do Multi-Agent LLM Systems Fail?" (Cemri et al., 2025) opens by noting that the performance gains of multi-agent systems on popular benchmarks are often minimal. The authors built a taxonomy, MAST, from over 1,600 annotated traces across 7 frameworks, with 14 failure modes in 3 categories:

Their interventions show that design matters and is not enough on its own. The authors report a 9.4% higher success rate for ChatDev from better role specifications alone, and up to 15.6% from adding a high-level verification step, while noting that not all failure modes were resolved and task completion stayed low. The practical lessons:

• Write termination conditions into every agent and enforce them in code: maximum rounds, maximum workers, maximum handoffs, a token budget and a wall-clock timeout.
• Detect loops: the same subtask, the same tool call or the same handoff repeated is a stop signal.
• Verify before you finish. A final check by tests, a schema or a separate evaluator catches what agents agreed on wrongly.
• Contain failures. A failed worker returns an error result, the lead decides what to do with it, and a partial answer says what is missing. Anthropic describes resuming from the point of failure with checkpoints rather than restarting a long run.
• Trace everything. Give the whole run one trace ID and each agent, model call and tool call a span with its parent, inputs, outputs, tokens and latency. See LLM observability and tracing. Anthropic’s write-up says adding full production tracing is what let it diagnose why agents failed.
• Evaluate the system, not just the parts. A run can fail with every agent doing its job; see how to evaluate AI agents.

Across process and company lines: A2A

Everything above runs inside one application. When agents built by different teams or vendors need to work together, the Agent2Agent protocol (A2A), originally developed by Google and now under the Linux Foundation, standardizes the exchange: an agent publishes an Agent Card, a JSON document describing its skills and endpoint; clients send it messages made of parts; work is tracked as a task with states such as working, input-required, completed and failed; and outputs come back as artifacts, with streaming and push notifications for long tasks, over JSON-RPC, gRPC or HTTP+JSON bindings. Agents stay opaque to each other, sharing no internal memory or tools. A2A covers agent-to-agent traffic, while MCP covers an agent’s access to tools. See what the A2A protocol is and what an MCP server is.

If you need several agents working together inside your product, BYD Tech builds AI agents: the code lives in your GitHub from day one, and AI-assisted work is reviewed by an engineer.