Streaming: incremental text and tool-call assembly

You could avoid all of this by not streaming. The model returns a complete message, you parse it, done. Nobody does this, for two reasons: latency (a user staring at a blank terminal for forty seconds will assume it hung) and interruptibility (you cannot interrupt work you cannot see).

The price is that the completion arrives as a sequence of fragments, and the fragments are not the unit of control flow. Text can be rendered as it arrives. A tool call cannot be dispatched as it arrives, because until the stream ends you do not know whether its arguments are complete.

dsh's wire vocabulary is a closed discriminated union, documented in docs/subsystems/llm-streaming.md:

type StreamChunk =
  | { type: 'block-start'; index: number; blockType: ContentBlockType }
  | { type: 'text-delta'; index: number; text: string }
  | { type: 'reasoning-delta'; index: number; text: string }
  | { type: 'tool-call-delta'; index: number; id: CallId; name?: string; argumentsDelta: string }
  | { type: 'block-end'; index: number; block: ContentBlock }
  | { type: 'usage'; usage: TokenUsage }
  | { type: 'finish'; reason: FinishReason; replayState?: ReplayEnvelope }

index correlates deltas to blocks — a completion can interleave two tool calls and a text block, and index is what keeps them apart. Two ordering rules from the adapter contract are load-bearing: usage arrives before finish, and nothing arrives after finish.

Here is the concrete failure. The model wants to call write_file with a 40-line body. What arrives on the wire is roughly:

{ type: 'block-start',      index: 1, blockType: 'tool-call' }
{ type: 'tool-call-delta',  index: 1, id: 'call_a7', name: 'write_file', argumentsDelta: '{"pa' }
{ type: 'tool-call-delta',  index: 1, id: 'call_a7', argumentsDelta: 'th": "src/ma' }
{ type: 'tool-call-delta',  index: 1, id: 'call_a7', argumentsDelta: 'in.ts", "content": "expo' }
{ type: 'tool-call-delta',  index: 1, id: 'call_a7', argumentsDelta: 'rt const x' }
   … 30 more fragments …
{ type: 'tool-call-delta',  index: 1, id: 'call_a7', argumentsDelta: ' = 1\\n"}' }
{ type: 'block-end',        index: 1, block: { type: 'tool-call', id: 'call_a7', name: 'write_file', arguments: '{"path": "src/main.ts", "content": "export const x = 1\\n"}' } }
{ type: 'usage',   usage: { inputTokens: 8123, outputTokens: 412 } }
{ type: 'finish',  reason: { kind: 'tool-calls' } }

Note that name appears on the first fragment and is absent afterwards, and that no single fragment is valid JSON. '{"pa' does not parse. 'th": "src/ma' does not parse. Naive implementations break in three characteristic ways:

1. Parsing per-delta. JSON.parse(chunk.argumentsDelta) throws on every fragment. The usual "fix" is a try/catch that swallows the error, which then silently drops the call.
2. Dispatching on the first fragment that happens to parse. With small arguments, '{"path":"a"}' can arrive whole in one delta, so this works in testing and then truncates a large payload in production — writing a half-file.
3. Concatenating without index. Two concurrent tool calls interleave their deltas, and you get one call with the other's arguments spliced into the middle.

The rule is: accumulate, then dispatch after the stream ends. In dsh the accumulator is BlockAssembler (packages/llm/llm/src/assembler.ts), which is the single shared fold from chunks back into content blocks. It is deliberately the only implementation of this fold in the codebase, because two implementations would drift and only one of them would be the one that persists.

The contract from docs/subsystems/llm-streaming.md that makes this tractable: tool-call arguments stay raw JSON strings end-to-end. Partial fragments stream via argumentsDelta; a provider that hands back parsed objects re-stringifies at block-end. The loop never has to reason about "is this an object yet?" — it is a string until something parses it, once, at dispatch.

The assembler is also defensive in two ways worth stealing:

• It is tolerant of delta-only protocols that never emit block-start/block-end, because not every provider does.
• Deltas arriving for an index already closed by block-end are ignored as a malformed stream, so a misbehaving adapter can neither grow memory unboundedly nor corrupt a completed block.

And one rule that surprises people: a max-tokens finish drops every tool call. A truncated tool call is unsafe to execute — the arguments JSON is by definition incomplete, and "best effort" on a write_file whose content was cut off means writing a truncated file. Dropping is the only safe policy.

The step loop does two things per chunk, and the order matters:

// packages/core/agent-loop/src/agent.ts (step)
const stream = preparedCall?.stream(request) ?? this.loopCtx.llm.stream(request)
signal.throwIfAborted()
for await (const chunk of stream) {
  signal.throwIfAborted()
  chunkSeqs.push(this.session.append('assistant/chunk', { turn, step, chunk }).seq)
  assembler.push(chunk)
}

Every raw chunk is appended to the session log as an assistant/chunk event and folded into the assembler. When the stream finishes, the assembled message is appended as a separate assistant/message event carrying sourceEventSeqs: chunkSeqs — a citation from the assembled artifact back to the raw evidence that produced it.

This looks redundant. It is not, and docs/subsystems/session.md makes the durability requirement explicit: the durable log persists every event losslessly, including assistant/chunk, and seq must stay contiguous, so chunks cannot be filtered out of the canonical log. You keep the raw chunks because token-level replay fidelity is what lets a UI reconstruct exactly what the user saw, and because when an adapter is suspected of malforming a stream, the chunks are the evidence.

This is the section's real lesson. If your renderer is driven by the same iteration that drives dispatch, you have coupled two things with incompatible requirements:

• The UI wants every fragment, immediately, lossily. Dropping a frame is fine.
• Control flow wants complete, validated, ordered blocks, exactly once. Dropping anything is corruption.

dsh separates them at the event layer. docs/agent-lifecycle.md states the rule for integrators directly: SDK users that need replayable transcript data should consume session/event; agent/* is the live coordination API. Two audiences, two channels, one source of truth. A terminal renderer subscribes to the live chunk stream and can drop, throttle, or coalesce freely. A transcript exporter folds session/event and gets exactly what happened.

The failure this prevents is specific and common: a harness whose control flow is entangled with rendering will deadlock or drop tool calls when the UI backpressures — a slow terminal, a paused scrollback, a websocket that stopped draining. Decoupling means the loop's progress never depends on a consumer's speed.

Logging raw chunks costs storage — a long session's log is dominated by assistant/chunk events. dsh accepts this and pushes the cost to the backend's encoding (the JSONL backend packs chunk rows; docs/subsystems/persistence.md covers it), rather than dropping data at the source. Telemetry makes the opposite choice: per docs/subsystems/session-telemetry.md, only the first assistant/chunk of each (turn, step) ships as a stream-started signal and the rest drop at capture, so seq gaps on the wire are routine and never a loss signal. That is the right call for an outbound reporting channel and the wrong call for the durable log.

Most harnesses assemble tool calls the same way — the fragmentation is a property of the provider protocols, not of any harness. The divergence is in what gets persisted. Many harnesses persist only the assembled message and treat the raw stream as ephemeral, which is cheaper and loses the ability to replay token timing or audit an adapter. Some persist a UI-oriented transcript that is neither the raw stream nor the model-visible history, which is the worst of both: it cannot reconstruct the request and it cannot reconstruct the display.