One level of fan-out, and why that is enough

A subagent cannot spawn a subagent.

Every multi-agent design I have built is downstream of that one sentence, and I spent a while treating it as a limitation before noticing it was doing most of my architecture for me.

The case this post is about#

One review, three dimensions, two stages each — and the wait nobody meant to insert:

barrier   :  wall clock = Σ (slowest item per stage)
pipeline  :  wall clock = slowest single item's whole chain

The natural way to write it is: run all three finders, wait, then run all the verifiers. That wait is a barrier, and it costs the difference between the slowest dimension and the fastest, every stage.

Nothing about the one-level constraint requires it. I had written it that way because the plan was written in stages, and stages are how plans read.

This post follows those two lines, because the constraint that actually shapes multi-agent design — subagents cannot spawn subagents — is not the thing that cost the wall clock.

The symptom#

The natural way to decompose a large task is recursively. Split it in half, hand each half to something that will split it again, and let the tree find its own depth. It is how you would organise people, and it is the first design anyone sketches.

It is not available. The worker you delegate to has no delegation of its own, so a plan that reads "the research agent will fan out to three sub-researchers" does not describe a system that can exist — it describes one where the research agent does all three itself, sequentially, in one context, which is the thing you were trying to avoid.

The failure is quiet, because the plan still runs. It just runs flat, in one context, at the wall clock of the sum rather than the maximum, and the reason is a constraint you designed around without checking.

Why the obvious fix failed#

The obvious fix is to make the orchestrator a subagent too — a small coordinator that receives the task and hands out work. That inverts the problem rather than solving it: a coordinator with no ability to delegate is just a worker with a plan.

This is a real rule in the framework I work in, and it is stated as a constraint on where code may run: orchestrators must run in the top-level session, never as a dispatched agent. Anything that needs to fan out has to live where fan-out is possible. There is one scheduler, and it is the session you are talking to.

The second attempt is to fake depth by chaining. Dispatch a stage, collect, dispatch the next stage from those results. This works — it is most of what I actually do — but it is not nesting, and pretending otherwise leads to the mistake in the next section, which cost me more time than the constraint ever did.

The actual mechanism#

Accept one scheduler, and design the work as stages of independent items.

top-level session   →  decompose into N items
                    →  fan out N workers (one level)
                    →  collect
                    →  synthesize

Three properties fall out, and they are the whole benefit of the constraint:

The scheduler is the only thing that holds the whole picture. Workers see their item and nothing else, which is exactly what makes them cheap. There is no risk of a deep tree where nobody knows what anybody else concluded.

Fan-out width is a real number you control, not an emergent property of how enthusiastically the tree expanded. Concurrency caps apply at one place.

Synthesis has an obvious home. With a tree you have to decide at which node results get merged. With one level, the scheduler merges, because there is nowhere else.

The fix#

The mistake worth writing down is not the nesting constraint. It is the barrier you insert without meaning to.

The natural way to chain stages is: run all of stage one, wait, run all of stage two. That wait is a barrier, and it costs the difference between the slowest item and the fastest one — every time.

The alternative is to let each item flow through all its stages independently:

barrier   :  wall clock = Σ (slowest item per stage)
pipeline  :  wall clock = slowest single item's whole chain

Item A can be in stage three while item B is still in stage one. Nothing about the one-level constraint requires the barrier; I had just been thinking in stages because stages are how the plan was written.

A barrier is genuinely correct in exactly one situation: stage two needs cross-item context from all of stage one. Deduplicating results before expensive downstream work. Early-exiting when the total count is zero. Comparing each finding against all the others.

The two shapes differ by where the await sits, and that is the entire diff:

type Stage<I, O> = (input: I) => Promise<O>;

// BARRIER: every item must finish stage N before any item starts stage N+1.
async function barrier<T>(items: T[], ...stages: Stage<any, any>[]) {
  let current: any[] = items;
  for (const stage of stages) {
    current = await Promise.all(current.map(stage));   // <- the wait
  }
  return current;
}

// PIPELINE: each item walks its own chain. Nothing waits for a sibling.
async function pipeline<T>(items: T[], ...stages: Stage<any, any>[]) {
  return Promise.all(items.map(async (item) => {
    let value: any = item;
    for (const stage of stages) value = await stage(value);   // <- per item
    return value;
  }));
}

A review over three dimensions whose stage-one durations differ by 3×, measured with a fake clock:

const DURATIONS = { bugs: [9, 4], perf: [3, 4], style: [3, 2] };   // [find, verify] seconds
const trace: string[] = [];
const stage = (n: 0 | 1) => async (key: keyof typeof DURATIONS) => {
  const start = clock.now; await clock.advance(DURATIONS[key][n]);
  trace.push(`${key} stage${n + 1}: ${start}s -> ${clock.now}s`);
  return key;
};

await barrier(["bugs", "perf", "style"], stage(0), stage(1));
await pipeline(["bugs", "perf", "style"], stage(0), stage(1));

barrier                        pipeline
bugs  stage1: 0s ->  9s        bugs  stage1: 0s ->  9s
perf  stage1: 0s ->  3s        perf  stage1: 0s ->  3s
style stage1: 0s ->  3s        style stage1: 0s ->  3s
perf  stage2: 9s -> 13s        perf  stage2: 3s ->  7s
style stage2: 9s -> 11s        style stage2: 3s ->  5s
bugs  stage2: 9s -> 13s        bugs  stage2: 9s -> 13s
total 13s                      total 13s

Same total, and that is the honest result for three items — the slowest chain dominates either way. The barrier's cost is in the two middle rows: perf and style sat idle from 3s to 9s waiting for a dimension they have nothing to do with. Widen this to the 84-item version and those idle windows are what you are paying for, which is why the shape matters at scale and looks free at three.

It is not justified by needing to flatten, map or filter between stages — that transform belongs inside a stage — and it is not justified by the stages feeling conceptually separate. Separate stages are not synchronised stages, and the latency is real.

How we proved it#

The honest test is what I actually did with this article's own batch, because I chose the slower shape on purpose.

Seven posts, each needing prose, five generated images, and a verification pass. That is a textbook fan-out: seven independent items, no cross-item dependency, one level deep.

I ran it serially, in one context, and the reason is a constraint that beats the topology:

image generation: serially, never batched

The producer is quota-bound and its account is pinned per project, so seven parallel workers would have contended for one resource and the failure mode is a partial batch with no clean retry point. The wall-clock win was available and the resource would not support it.

What I did fan out was the reading — the independent question "is this rendered image any good" — which is stateless, needs no shared resource, and is exactly the shape the one-level design wants.

The honest limitation, and it is structural: one scheduler is one context, and contexts fill. The whole argument for fanning out is that workers keep their reading out of the scheduler's head — but the scheduler still holds every result it collects, every decision it made about decomposition, and the synthesis at the end. A long enough run exhausts the one thing that cannot be delegated. The mitigations are real but partial: return shapes rather than prose, summarise at collection rather than at synthesis, and persist state to disk so a fresh context can resume.

What the constraint is actually protecting#

I have come round to thinking the restriction is load-bearing rather than incidental.

An unbounded tree of delegating agents has no natural limit on cost, and the cost is not visible from any single node. Each level looks like a reasonable local decision to split work; the total is a multiplication nobody authored. A tree three deep with a fan of four is sixty-four leaves, and the node that caused it made one call.

One level makes the bill legible. The scheduler knows it is dispatching N, so N is a number that appears in the design, gets a cap, and shows up in a log. That is a property I would want even if recursion were on offer.

What this generalizes to#

Design for one scheduler and one level, because that is what exists. Decompose at the top, fan out the independent items, and synthesize where the whole picture already lives.

Then check your stage boundaries for barriers you did not intend — let each item run its whole chain unless a later stage genuinely needs all the earlier results at once. And before reaching for parallelism at all, check what the work actually contends on: a shared quota, a single account, or an unretryable write will beat the topology every time, and the serial version you can resume is worth more than the parallel one you cannot.