Certified Agent Harness Engineer
Course sections

M3.4

The tool execution pipeline

798 words · about 5 min

Concept

Between "the model asked" and "the tool ran" there is a sequence of policy decisions, and the order is not arbitrary. Getting it wrong produces the two classic harness security bugs: a permission check that a later hook can undo, and a tool that runs before the thing that was supposed to authorize it.

Mechanism

dsh's pipeline, in order (docs/tool-execution-pipeline.md):

tool/call session event (logged BEFORE execution)
      ↓
tools/pre-execute        waterfall — allow | deny | ask
      ↓
ctx.approval             one-shot prompt (only on 'ask')
      ↓
registered guards        monotonic — deny or abstain, never allow
      ↓
tools/execute            waterfall — around dispatch (timeout, retry, metrics)
      ↓
   tool body execute()
      ↓
tools/post-execute       waterfall — accept | replace | block | add context
      ↓
registry normalization   throws and invalid values become isError
      ↓
finalizeContent          definition-owned, synchronous, content-only
      ↓
tools/result             emit — frozen authoritative outcome
      ↓
tool/result session event

Two structural properties deserve close attention.

The pipeline is monotonic at the security boundary. tools/pre-execute is a reorderable waterfall returning PreToolDecision — allow, deny, or ask. Because listener order is configurable, a pre-execute listener alone cannot be trusted with final authority. So dsh adds registered guards after it, and gives them a deliberately impoverished return type:

type ToolGuard = (execution: Readonly<ToolExecution>) => string | undefined

There is no allow result. A guard can return a denial reason or abstain with undefined. This is the whole trick: since no guard can produce permission, no amount of listener reordering can turn a denial back into an allow. Authority flows one direction only. Write that down — it is the single most exam-relevant structural fact in this module.

Every failure path converges. A denial, a throwing tool, an unknown tool, a timeout, an invalid return value, a throwing post-execute listener — all become a normalized isError result that still reaches finalizeContent and tools/result. There is no path where a tool call vanishes without producing an outcome. ToolNotFoundError maps to UNKNOWN_TOOL "so the call fails without ending the turn" — an unknown tool is a recoverable model mistake, not a fatal harness condition.

Note also that ask degrades to deny when no approval service is mounted. Fail-closed by default; a missing dependency must never become an implicit grant.

Worked example — the shape of an around-dispatch wrapper

Worked example

Module 2 introduced the timeout policy as a cancellation example. Read it again as a pipeline example, because it demonstrates the exact contract a tools/execute wrapper must honor:

// packages/guard/timeout-policy/src/index.ts
ctx.on('tools/execute', async (exec, next): Promise<ToolExecutionResult> => {
  const timeoutMs = ctx.tools.get(exec.name, exec.agent)?.timeoutMs
  if (timeoutMs === undefined) return next()
  using d = deadline(exec.signal, timeoutMs, TOOL_TIMEOUT)
  const upstream = exec.signal
  exec.signal = d.signal
  try {
    const result = await next()
    if (timeoutOf(d.signal, TOOL_TIMEOUT) !== undefined) return toolTimeoutResult(timeoutMs)
    return result
  } finally {
    exec.signal = upstream
  }
})

The wrapper may replace exec.signal — and only exec.signal. Identity fields are readonly. It must restore the original in a finally. And critically, the registry re-fuses the original caller signal before invoking the body, so a buggy or malicious wrapper cannot detach caller cancellation by substituting a signal that never aborts. The extension point is powerful but bounded by construction, not by convention.

Tradeoffs

Deep extensibility costs latency and, more importantly, comprehensibility: with three waterfalls plus guards plus a finalizer, "why did this call get denied?" becomes a real debugging problem. The Module 1 spine's if (name === 'read') is trivially traceable. The justification for the pipeline is that policy — permissions, sandboxing, timeouts, telemetry, retry — must be composable across tool families without every tool knowing about every policy. The cookbook states the design rule directly: "Prefer not to build deployment policy into the tool."

How `dsh` does it

How dsh does it

The pipeline is a Cordis waterfall chain, so policy ships as ordinary plugins. This is the AGENTS.md invariant plugins, not loop changes applied to tools: a new permission model, a new sandbox, a new metrics sink — none of them touch the agent loop or any tool. Note the deliberate naming distinction that trips people up: tools/result (plural) is the live in-process event, while tool/result (singular) is the durable session event the loop appends afterward.

How others differ

ACP inverts the arrangement entirely. Under the Agent Client Protocol the agent runs as a subprocess of the editor and calls session/request_permission back to the editor — the editor owns the permission UI and the decision. Likewise fs/read_text_file and fs/write_text_file let the editor serve file contents, so the agent sees unsaved buffers. In dsh terms, ACP relocates the pre-execute gate across a process boundary and hands it to the client. That is a genuinely better fit when a human is watching an IDE, and a worse fit for headless automation where no client exists to answer.


Check yourself

3 questions on M3.4

Start drill