Token metering and per-turn accounting

Every loop iteration adds to the context. Token accounting is not a billing feature; it is a control input. Compaction needs to know when pressure is high. The UI needs to show the user how much room is left. Cost reporting needs per-turn attribution. All three read the same measurement, and they will disagree unless there is exactly one.

TokenUsage in dsh carries five fields, and the rule that governs them is the one people get wrong:

interface TokenUsage {
  inputTokens: number        // UNCACHED input only
  outputTokens: number
  cacheReadTokens?: number
  cacheWriteTokens?: number
  reasoningTokens?: number   // informational — ALREADY INCLUDED in outputTokens
}

The counts are disjoint. Billed input is the sum of inputTokens, cacheReadTokens, and cacheWriteTokens — inputTokens is uncached input only, so adding cache fields to a value that already included them double-counts. And reasoningTokens is already included in outputTokens; totals must not add it again. Both mistakes produce plausible-looking numbers that are wrong by exactly the amount that matters when a user is deciding whether a run is affordable.

Usage arrives as a usage chunk before finish, and is stored two ways. The assistant/chunk event preserves it raw. The assembled assistant/message event carries the step's usage when the adapter reported it, so the model output and its accounting travel together — there is no separate usage record.

That co-location matters for failed attempts, and docs/subsystems/session.md spells out the rule: token accounting reads per-step assistant/chunk { type: 'usage' } records and treats assistant/message.usage as the committed-step fallback when no usage chunk exists. Failed model-request attempts have no assistant message, so their usage chunk is the durable accounting record. A retried request that failed after consuming 8,000 input tokens still cost 8,000 tokens; a system that only read assistant/message.usage would under-report every retry, and retries are exactly when costs spike.

ctx.tokenMeter.measure(session, requestHeader?) returns a TokenMeasurement (packages/llm/token-meter/src/types.ts) with:

• logRevision — how many durable events the measurement consumed, so a consumer can tell a stale measurement from a current one.
• baseline — either 'usage' (anchored to a real provider count) or 'estimated' (anchored to a heuristic). The meter uses provider usage when the latest successful call's canonical request envelope matches and its total is no lower than that call's heuristic anchor, and falls back to estimation otherwise.
• surfaceDeltaTokens — a signed repricing of current surface content relative to that anchor.
• totalTokens — request-and-response pressure, computed as max(0, baseline.tokens + surfaceDeltaTokens).
• nodes — per-surface-node { seq, tokens } in positional head-to-tail order.

The baseline-plus-delta structure is the design. A pure estimate is always available but wrong by a model-specific tokenizer margin. A pure provider count is exact but only available after a request and only for what that request contained. Anchoring to the last real count and pricing only the delta since gives you the accuracy of the provider number for the bulk of the context and the availability of the estimator for the recent tail.

estimateMessage(message) is exposed separately (packages/llm/token-meter/src/estimate.ts) for the "would this fit?" question that must be answered before any request exists.

A caution the docs state and that trips people up: surface order is authoritative, and replacement nodes can have higher durable seqs than later positional nodes. After a compaction rewrites part of the surface, seq order and positional order diverge. Code that sorts nodes by seq to compute a running total will be wrong on any compacted session.

Estimation is approximate by construction — dsh uses a heuristic estimator rather than shipping a real tokenizer per provider. Bundling tokenizers would mean vendoring large model-specific tables that go stale on every model release, for a number that is only ever used to decide when to act, not what to bill. The honest position: the estimate is good enough to trigger compaction and to draw a progress bar, and it is not good enough to bill a customer. Use the provider's reported usage for money.

Claude Code displays live context usage and triggers auto-compaction from it — same signal, same purpose. The main divergence across harnesses is what counts as pressure: dsh measures request-and-response pressure over the model-visible surface, while some harnesses count only the message array and under-report by the size of the system prompt and tool schemas, which on a large tool catalog can be thousands of tokens. Anything that reaches the request should be counted, which is again the model-visible ⟺ logged invariant paying off — if it is logged, it can be measured.

Hands-on lab — refactoring the Module 1 spine

Take your Module 1 spine and evolve it in six commits. Do not import dsh; the point is to build the shapes yourself so that reading packages/core/agent-loop/src/agent.ts afterwards feels familiar.

Lab 1 — Name the exit conditions. Replace the single if (calls.length === 0) break with a StepOutcome discriminated union covering at least completed, max-tokens, aborted, and error, and a runStep() that returns StepOutcome | null where null means continue. Split runTurn() from runStep(). Make max-tokens sticky. Success: a completion with finish_reason: 'length' ends the turn as max-tokens even if a later step completes normally.

Lab 2 — Stream and assemble. Switch to a streaming request. Write an assembler keyed by block index that accumulates argumentsDelta fragments into a string per tool call and only produces the call at block end. Render text deltas to stdout as they arrive. Success: a tool call whose arguments exceed 2 KB dispatches with complete arguments. Verify by logging every fragment and confirming none of them individually parses as JSON. Then drop every tool call when the finish reason is max-tokens and confirm nothing dispatches.

Lab 3 — Cancel. Add an AbortController per turn wired to SIGINT. Call signal.throwIfAborted() before the stream, on every chunk, and at each step boundary. On abort, finalize the delivered text prefix with an interrupted: true marker and drop any assembled tool calls. Give every undispatched call a synthetic error result so no call lacks a result. Success: Ctrl-C during a long stream stops within one chunk; the transcript has no tool call without a matching result; re-sending the transcript to the provider is accepted.

Lab 4 — Guard. Add a repetition detector that canonicalizes arguments by deep key-sorting before stringifying, counts consecutive identical (name, args) calls per agent, injects an advisory reminder at counts 3, 5, and 8, resets on any user message, and never blocks a call. Add a per-tool timeout that derives its signal from the caller's, awaits rather than races, and replaces the result only when its own timer fired. Success: a tool that ignores its abort signal still returns a TOOL_TIMEOUT result and is awaited to completion — no orphaned promise; and {b:2,a:1} after {a:1,b:2} counts as a repeat.

Lab 5 — Persist and resume. Replace the in-memory messages array with an append-only JSONL event log using event types modeled on turn/start, turn/end, step/start, step/end, user/message, assistant/chunk, assistant/message, tool/call, tool/result. Derive the message array by folding the log, skipping assistant/chunk and empty-content assistant messages. Recover the turn counter with a findLast over turn/start. On load, close any turn/start with no turn/end as interrupted. Success: kill -9 mid-turn, then restart: the turn counter continues correctly, the orphaned turn is closed as interrupted, and the derived message array is a valid provider request.

Lab 6 — Meter. Capture the usage chunk into the log, store it on the assembled message, and compute per-turn totals by folding. Keep inputTokens disjoint from cache fields and do not add reasoningTokens to output. Print a per-turn line: uncached input, cache read, cache write, output, and a running context estimate. Success: a turn with three steps and one failed-and-retried request reports the failed attempt's tokens. Deliberately delete the failed attempt's usage from your accounting and observe the under-report — that is the bug this design prevents.

Common pitfalls

1. Dispatching a tool call before the stream finishes. Symptom: works in testing, truncates large payloads in production. Fix: dispatch only after finish; treat arguments as an opaque string until then.
2. JSON.parse on argumentsDelta, or throwing on unparseable arguments. Symptom: silently dropped calls, or a turn killed by a model typo. Fix: accumulate fragments; on parse failure preserve the raw string and let schema validation return a normal error result the model can correct (parseArguments in packages/core/agent-loop/src/tool-calls.ts).
3. Keeping a tool call in an interrupted assistant message. Symptom: transcript has a call with no result; providers reject the next request. Fix: on interrupt, salvage only text and reasoning; drop tool calls; give undispatched calls a synthetic error result so every call has a result.
4. Promise.race for tool timeouts. Symptom: the loop moves on while a subprocess keeps writing files. Fix: derive a chained signal, await the tool, and replace the result only when your own timer fired — checked by a scoped code so a nested outer deadline is not misread as yours.
5. Retry state in memory. Symptom: a crash during backoff restarts the retry count at zero and hammers a provider that already asked you to wait. Fix: log the scheduled retry before the wait, recover the count from the log.
6. Retrying a context-window error. Symptom: identical request, identical failure, until the retry budget is gone. Fix: canonicalize the code (CONTEXT_WINDOW_EXCEEDED), exclude it from retryableCodes, and route it to compaction on the same recovery extension point.
7. Double-counting cached input, or adding reasoning tokens to the total. Symptom: cost reports inflated by 2× on a cache-heavy workload. Fix: treat inputTokens as uncached-only; treat reasoningTokens as informational because it is already inside outputTokens.
8. Sorting token-surface nodes by seq. Symptom: correct on fresh sessions, wrong after compaction. Fix: surface order is authoritative; a replacement node can carry a higher seq than a later positional node.
9. Injecting model-visible context without logging it. Symptom: resume produces a different request than the original run; a bug is unreproducible. Fix: uphold model-visible ⟺ logged — every guard reminder, injected file notice, and system prompt section that reaches the request is a session event, with a source label so a transcript does not attribute plugin text to the user.
10. Coupling the renderer to the dispatch loop. Symptom: a paused terminal or slow websocket stalls the agent. Fix: two channels — a lossy live stream for display, the durable event log for anything that must be exact.

Key terms

• Step — one model request plus the tool calls it produced. One trip around the canonical loop.
• Turn — one user-facing unit of work; one or more steps, opened by turn/start and closed by turn/end with a typed reason.
• Exit condition — a named reason a turn stops: completed, max-tokens, aborted, blocked, error, plus interrupted synthesized by crash recovery.
• Sticky outcome — an outcome that cannot be downgraded by later steps; max-tokens is the example.
• Stream chunk — one wire fragment: a block start/end, a text/reasoning/tool-call delta, usage, or finish.
• Block assembly — folding index-correlated deltas back into complete content blocks before dispatch.
• Interrupted prefix — the text and reasoning an aborted stream can safely finalize; tool calls are excluded because interruption precedes dispatch.
• Cooperative cancellation — signaling intent via an AbortSignal and relying on the work to reach a consistent stopping point; the only kind available.
• Drain — awaiting settlement of in-flight work after an abort rather than abandoning its promises.
• Guard — a plugin that watches the loop for unproductive patterns or enforces per-call budgets; advisory or replacing, not a swappable capability.
• Canonicalization — deep key-sorting before stringifying so semantically identical arguments compare equal.
• Structured failure — a typed failure record (message, code, status, retry-after, request id) rather than a stringly-typed exception.
• Canonical error code — one harness-owned code (e.g. context-window-exceeded) that every adapter maps its provider's wording onto, absorbing provider drift.
• Event log — the append-only, lossless, contiguously-sequenced record that is the source of truth.
• Projection — a pure synchronous fold from events to a value, versioned so stale cached state is discarded rather than forward-applied.
• Whole-value event — a state-carrying event that records the complete post-change state rather than a delta.
• Derived history — the model-visible message array folded from the log; raw chunks and empty assistant messages are skipped.
• Model-visible ⟺ logged — the invariant that anything reaching a model request must be reconstructable from the log.
• Disjoint usage counts — uncached input, cache read, cache write, and output are non-overlapping; reasoning tokens are already inside output.
• Baseline and surface delta — anchoring a token measurement to the last real provider count and pricing only the change since.

Summary — what changed in your mental model

Before this module, the loop was a while with one exit. After it:

The loop is a state machine with an enumerated exit vocabulary, not a while with a break. Six named outcomes, one of which (interrupted) no loop ever emits because it belongs to crash recovery. If you cannot name why your loop stopped, your users cannot either.

Fragments are not the unit of control flow. Text renders incrementally; tool calls dispatch only after assembly. The UI and the loop read the same stream through different contracts, and coupling them makes the agent's progress hostage to a consumer's speed.

Cancellation reaches into tools or it does not exist. A signal you can only check between tools is not cancellation. Chain the signals, check at every boundary, drain rather than abandon, and give every logged call a result — including a synthetic one — because a transcript with an unanswered call is corrupt.

Failures are data. A structured failure with a canonical code can be classified, routed, and recovered from by policy. A string message can only be regexed, and the regex breaks when the provider rewords the error.

The log is the source of truth; everything else is a projection. Resumption stops being a feature you build and becomes a consequence of the design — the turn counter is a findLast over the log. That property only holds if nothing model-visible can bypass the log, which is why the invariant is stated as an invariant rather than a guideline.

Token accounting is a control input. Compaction, UI, and cost reporting must read one measurement, and it must count failed attempts, because failures are when spending spikes.

Where this field is genuinely unsettled, and you should not accept confident claims otherwise: general no-progress detection. Repeated-identical-call detection is solved and shipped. Detecting a model that is thrashing productively-shaped but useless work is not solved by anyone, and dsh does not claim to solve it either — its shipped answer is an advisory reminder that never vetoes, which is an honest response to a heuristic that will sometimes be wrong.

Further reading

Repo — read in this order:

• docs/agent-lifecycle.md — the generated sequence diagram of the full turn/step lifecycle. Generated, not hand-edited; regenerate with pnpm run gen-doc-graphs.
• packages/core/agent-loop/src/agent.ts — ReactLoopAgent; the Phase union, kick/turn/step, preStep, and buildRequest.
• packages/core/agent-loop/src/tool-calls.ts — dispatch scheduling, barriers, the bounded rolling pool, and abort handling.
• docs/subsystems/llm-streaming.md — StreamChunk, BlockAssembler, LlmFailure, TokenUsage, retry policy shapes.
• docs/subsystems/session.md — SessionEventMap, TurnEndReasonMap, derived history, and the durability contract.
• docs/subsystems/session-projection.md — the projection contract, stateVersion, checkpoints, and the cold-read ladder.
• docs/subsystems/token-meter.md and packages/llm/token-meter/src/types.ts — TokenMeasurement and its baseline.
• packages/guard/README.md, packages/guard/repeat-tool-reminder/src/index.ts, packages/guard/timeout-policy/src/index.ts — the two shipped guards.
• packages/llm/llm-retry/src/index.ts — durable, cancellable, provider-routed backoff.
• AGENTS.md — the invariants this module keeps citing, especially model-visible ⟺ logged and plugins, not loop changes.

External:

• The WHATWG DOM standard's AbortController/AbortSignal section — the cancellation primitive, including AbortSignal.any() for fusing signals and throwIfAborted().
• Provider streaming documentation from Anthropic, OpenAI, and DeepSeek — read all three and note that the fragmentation shape is common while the event names are not; this is what an adapter layer absorbs.
• Martin Fowler's "Event Sourcing" and "CQRS" write-ups — the event-log-plus-projection pattern predates agents by twenty years, and the tradeoffs are the same ones.
• Claude Code and OpenAI Codex CLI documentation on session resumption and interrupt behavior — useful for seeing which of these choices are universal and which are dsh's.

Reinforcement questions for this module: modules/02-agent-loop-questions.md.