The position, stated up front
The corollary matters just as much. The orchestrator’s job is bookkeeping, not intelligence. Most of the value in a working hive comes from durable state, typed handoffs and retries, and none of that requires a model at all.
Why the big agent fails
Take a process with twelve steps. Suppose each step is right 95% of the time, which is a good agent on a real task. If the steps are dependent, and in a business process they nearly always are, the probability of a clean end-to-end run is 0.95 raised to the twelve. That is roughly 54%.
Half your runs go wrong somewhere, and here is the part that hurts: you cannot tell where. A single agent working through a long context produces one output at the end. If it is wrong, the failure could be in step three or step nine, and diagnosis means reading a transcript.
Split the same job into four units with checkable outputs and the picture changes. Each unit’s output is validated before the next begins. A failure is caught at the unit that caused it, retried in place, and never propagates. You have not made the model better. You have made failures local.
There is a second effect, and it is the one people feel before they can name it. Context does not scale linearly with usefulness. A prompt holding twelve steps of instruction, four tool schemas and the accumulated output of the previous eight steps is competing with itself. Instructions that mattered at step two are still sitting there at step ten, diluting the ones that matter now. Smaller scope means a smaller, sharper context budget per call, which is cheaper and more accurate at the same time.
The hive, and what it maps to
The metaphor earns its place here because the structure is genuinely the same shape.
A hive does not have one bee doing everything. It has scouts, foragers, nurses and guards, each with a narrow job, coordinated by signals rather than by a manager watching. In our systems the same split is literal: each agent owns one task, and a coordination layer routes work between them by passing typed messages through a queue.
That coordination layer is the orchestrator. It holds the plan, the task queue, the state of every in-flight job, and the rules for what happens when something fails. If you deleted every agent and kept the orchestrator, you would still have most of the engineering.
What is multi agent orchestration made of
Six responsibilities. Only the first involves a model, and only sometimes.
Handoffs: contracts, not conversations
Agent handoff patterns are where designs go wrong most often, so this deserves a direct opinion.
Pass a typed object instead. The producing agent emits a structure with declared fields, the structure is validated against a schema, and the consuming agent receives data rather than prose. If the object fails validation, that is a caught error at a known boundary, not a confusing result four steps later.
Three patterns cover almost everything we build:
The chain pattern is also the main cost lever, which is covered in what AI agents actually cost.
- Supervisor - One coordinator holds the plan and calls specialists in sequence. Best when the order is known and the steps depend on each other. Easiest to audit.
- Queue and claim - Tasks land in a queue, agents claim what matches their scope, results return to the queue. Best for high-volume, independent work such as ticket triage or document processing. Scales by adding workers.
- Escalation chain - A cheap, fast agent handles the common case and passes the rest up to a more capable and more expensive one, with a human at the top. Best when case difficulty is uneven, which is most support and back-office work.
Failure modes with names
Multi-agent systems fail in ways single agents do not. Five that we design against by default.
Every one of those is a distributed-systems problem that predates language models by decades. That is the point. If your orchestration vendor talks about reasoning and not about leases, retries and idempotency, they have not run this in production.
What it looks like when it works
An alert fires at 03:14. The orchestrator opens a job and dispatches the first task to the Incident Response Agent, which gathers evidence, correlates recent deploys and forms a probable cause. That output is a typed incident object, not a paragraph.
The object routes to a remediation stage, which checks the proposed action against the approved action list. The action is on the list, is reversible, and is inside the blast radius the policy allows, so it runs and the result is verified. If the check had failed, the job would have paged a human with the evidence attached rather than trying something creative at 3 a.m.
Both stages wrote to the same durable job record. If the process running stage two had died mid-action, the lease would have expired, the task would have returned to the queue, and the idempotency key would have stopped the action from happening twice. Nobody would have needed to know.
That is the whole trick. A hive that finishes work is a hive whose every step is checkpointed, verifiable and repeatable without harm. We build that layer as multi-agent orchestration.
How many agents is the right number
There is a real cost to each stage, so the answer is not “as many as possible”.
Every agent you add brings a scope definition, an eval set, a tool boundary, a set of permissions and a handoff contract to maintain. Every handoff adds latency and one more place a typed object can fail validation. Split too finely and you spend your engineering budget on plumbing between units that were never independently useful.
The test we apply is simple: a stage earns its own agent when its output is independently checkable and its failure is independently fixable. If you cannot write a validation rule for what comes out of a stage, that stage is not a boundary. It is the middle of a task and should stay inside one.
Applied to a document intake process, that usually gives four or five units rather than twelve. Extract. Classify. Enrich against your systems. Decide. Post. Each output can be inspected on its own, each failure points at one unit, and each unit can be improved without retesting the others.
Watching a hive you cannot see
Observability is not a nice-to-have here, because a multi-stage job has no single place to look.
Three things get instrumented from the start. A trace per job, showing every stage, retry, tool call and model version in order. Cost attributed per stage, so a runaway is attributable rather than mysterious. And queue depth with age, because the earliest sign of trouble in a hive is not an error rate. It is work sitting still.
Without those three, diagnosing a job that finished wrong means reading transcripts and guessing, which does not scale past the first few incidents.
When you do not need this
An honest limit, because orchestration is easy to oversell.
If your process is one task, one agent and one system, orchestration is overhead. Build the agent, give it a queue, and skip the coordination layer entirely. You need orchestration when work crosses stages that fail independently, when a job outlives a single request, or when different steps genuinely need different tools and permissions.
Adding agents also has a cost curve. Each one is a scope to define, a set of evals to maintain and a boundary to govern. Two well-scoped agents beat six vaguely scoped ones, every time. Start with the single process that hurts, which is the subject of choosing the first process to hand to an agent.
The short version
- Reliability compounds downward, so long single-agent jobs fail in the middle and cannot be diagnosed.
- Splitting work into units with checkable outputs makes failures local, retryable and cheap.
- The orchestrator’s real work is state, dispatch, retries, idempotency and escalation.
- Hand off typed objects between agents, never free text. An unvalidated interface fails silently.
- Multi-agent failure modes are classic distributed-systems bugs, and they need classic fixes.
- If the job is one task in one system, you do not need any of this yet.
Keep reading
- How to choose the first process to hand to an agent - Before you orchestrate anything, pick the right first job.
- What AI agents actually cost (per task, not per seat) - Where the money goes once several agents are working the same job.
- Questions, answered - Cost, safety, timelines and what stays human.
