Orchestration topologies

Three shapes cover almost everything.

Sequential. Each step's input is the previous step's output. Latency is the sum. Fully observable; simplest to debug; no concurrency benefit.

Fan-out / barrier. Start N children, wait for all, combine. Latency is the maximum branch, not the mean — the property most often mis-estimated. Five branches averaging 30 seconds, one of which takes four minutes, is a four-minute step. And the failure question is unavoidable: does one failed branch fail the whole barrier, or does it yield a hole in the results?

Pipeline. Items flow through stages; stage 2 can start item 1 while stage 1 works on item 2. Latency approaches the slowest stage rather than the sum of stages, and no full barrier is required. Best throughput, hardest to reason about, and results arrive out of order unless you enforce order.

Overlay two constraints on all three. Concurrency caps are mandatory — unbounded fan-out will exhaust rate limits, memory, or subprocess handles, and "N children" is chosen by a model that has no idea what N costs. And partial failure must have a stated semantics, chosen deliberately per topology.

The most useful idea in dsh's workflow engine is that not all failures are the same kind of failure, and the distinction is encoded in the type. WorkflowError.fatal marks errors that parallel() and pipeline() re-throw rather than mapping to a per-item null. Per-item null is reserved for an ordinary failed child run.

The reason: a child that failed to answer is data — one branch of the fan-out has no result, and the combiner can decide what that means. A workflow that exceeded its total agent budget, or whose script violated the value boundary, is not data; continuing past it produces a result that looks complete and is not. Collapsing both into null means a workflow that hit its budget cap on item 3 returns a partially-filled array indistinguishable from one where item 3's child simply failed.

The script surface (packages/workflow/workflow-worker-thread) is deliberately small:

• agent(prompt, { label, phase, schema, model }) — returns the structured value when a schema is supplied, final text otherwise; an ordinary failed child yields null.
• parallel(thunks) — runs under the configured concurrency limit.
• pipeline(items, ...stages) — passes (previous, item, index) without a cross-stage barrier.
• phase(title), log(message) — observability.

That pipeline note is the whole M6.5 lesson in one line of documentation. The absence of a cross-stage barrier is what makes it a pipeline; add the barrier and you have a sequence of fan-outs, with the max-not-mean cost at every stage.

Twelve files to analyze, each child averaging 40 seconds with a long tail (one file is large: 200 seconds).

Sequential: ~8 minutes. Fully ordered, trivially debuggable.

Fan-out with barrier, concurrency 12: 200 seconds. Eleven children idle for over two and a half minutes while the twelfth finishes. Real speedup — 2.4× — and 11 × 160 seconds of wasted wall-clock parallelism that a scheduler could have spent on something else.

Fan-out with barrier, concurrency 4: three waves, each costing its own max. If the large file lands in wave one: roughly 200 + 60 + 60 ≈ 320 seconds. The concurrency cap did not just cap resource use; it added a barrier per wave.

Pipeline, two stages (analyze → summarize): the summarize stage starts on file 1 while analyze works on file 2. Total approaches max(stage_total) plus one stage's latency, and no wave-level barriers exist at all.

Two conclusions students consistently miss. First, the tail dominates — reducing average child latency does almost nothing to a barrier; reducing the worst child does everything. Second, concurrency caps multiply barriers, so the interaction between cap and tail is where fan-out performance actually lives.

Sequential is slow and comprehensible. Fan-out is fast and pays the tail. Pipelining is fastest and hardest to debug: with no barrier, a bug in stage 2 manifests while stage 1 is still producing, and the interleaved log is not a story. Concurrency caps trade throughput for stability and are non-negotiable in production. Fail-fast barriers waste completed work; fail-soft barriers hand the combiner incomplete data it must be written to handle.

ctx.workflowEngine — one engine per context, unlike the named multi-provider subagent registry. WorkflowStartRequest carries { script, meta, args?, subagentProvider?, maxTotalAgents?, parent, signal? }; WorkflowResult is { value, stopReason, error?, agentsStarted }. WorkflowStopReason is a closed union — completed | cancelled | error — a deliberate contrast with the merge-extensible subagent stop reason, because plugins may add child kinds but not workflow outcomes.

Config in the worker-thread engine: maxConcurrentAgents (0 meaning resolve from CPU parallelism), maxTotalAgents (1000), maxItemsPerCall (4096), syncTimeoutMs (5000), disposeGraceMs (5000). Two documented honesty notes deserve emphasis. First: "The worker/vm is not a security boundary" — node:vm inside a worker is an API-shaping mechanism, and a workflow script is trusted code. Second: "Termination can only report host-observed starts" — agentsStarted excludes worker-side calls still queued behind concurrency, so it is a lower bound, not a count.

The model-facing tool (packages/workflow/tool-workflow, config maxResultChars default 50000) carries a limitation you must design around: "The parent turn blocks until the whole workflow settles — there is no background start/poll API, and cancellation discards partial output as an error." A workflow is a very expensive single tool call.

LangGraph is the best-known general graph orchestrator, and the honest statement here is that its specific API surface is not verified in this course's research and will not be examined. Claude Code parallelizes read-only tools within a turn — a within-turn analogue of fan-out that avoids the session overhead entirely, and often the right answer when "parallelism" is what you actually wanted. Goose subrecipes and OpenAI's Agents SDK handoffs sit at opposite ends: composition-by-inclusion versus transfer-of-control.