Budget a context window

A context window is not storage. It is a per-request budget that you re-spend from zero on every single step. Module 1 established that models are stateless: the harness resends the entire conversation each time. That makes the window a recurring cost, not a one-time allocation, and it makes "how full is it?" a question you must be able to answer before you send, not after the provider rejects you.

The budget has four categories, and only one of them grows without bound:

1. System prompt — harness identity, deployment persona, per-tool guidance. Roughly fixed per deployment.
2. Tool schemas — the JSON descriptions of every advertised tool. Fixed per tool set, but larger than most engineers guess.
3. History — user messages, assistant messages, and tool results. This is the one that grows.
4. Headroom — the space you deliberately do not spend.

Headroom is the category engineers omit, and omitting it is the single most common context bug. Two things must fit in it. First, the response: if the model is allowed 8,192 output tokens and you have filled the window to within 2,000 tokens of its limit, the model will be truncated mid-sentence or the request will be rejected outright, depending on the provider. Second, and less obviously, the next tool result. Consider an agent at 98% window occupancy that decides to call read on a source file. The request fits. The response fits. Then the tool returns 40KB of file content, that result is appended to history, and the next step's request does not fit. You have now failed at a point where recovery is harder, because a tool call is outstanding and the history is in an unbalanced state. Headroom must cover the response plus the largest plausible tool result, because a step is not complete until its observation is back in the history.

To budget, you must measure. There are two ways, and mature harnesses use both.

Heuristic estimation prices content locally, before sending. It is cheap, synchronous, and available for content that has never been sent. It is also approximate. dsh uses a fixed-density heuristic in packages/llm/token-meter/src/estimate.ts:

const CHARS_PER_TOKEN = 4
const BLOCK_OVERHEAD = 4
export const ROLE_OVERHEAD = 4

Text and reasoning blocks price as ceil(text.length / 4) + 4. A tool call prices its name and its serialised arguments separately, each at four characters per token, plus one block overhead. A tool result recurses into its own content blocks and adds a block overhead on top. Every message then adds ROLE_OVERHEAD for role framing:

export function estimateMessage(message: Message): number {
  return estimateContent(message.content) + ROLE_OVERHEAD
}

The envelope prices the same way — estimateSystemTokens is ceil(header.system.length / 4) + ROLE_OVERHEAD, and estimateToolsTokens is ceil(JSON.stringify(header.tools).length / 4) + BLOCK_OVERHEAD.

Provider usage anchoring replaces guesswork with truth for content already sent. Providers return actual token counts, and the harness records them. In dsh the canonical shape is TokenUsage in packages/llm/llm/src/types.ts:

export interface TokenUsage {
  inputTokens: number
  outputTokens: number
  cacheReadTokens?: number
  cacheWriteTokens?: number
  reasoningTokens?: number
}

The counts are disjoint by convention: inputTokens is uncached input only, and cached input is reported separately. This matters for accounting sanity, and it is a real translation burden on adapters. DeepSeek's wire format folds cache hits into prompt_tokens, so packages/llm/llm-deepseek/src/translate.ts subtracts them back out:

inputTokens: usage.prompt_tokens - (cacheRead ?? 0),

Get this wrong in either direction and you double-count or under-count the prompt side by exactly the cached portion — which, on a warm cache, is most of it.

dsh combines the two into a TokenMeasurement whose baseline is tagged 'usage' when anchored to a real provider count and 'estimated' when not, with a heuristic delta applied for everything appended since. That is the honest structure: exact where exactness is available, estimated at the frontier.

Take a 128,000-token window — a common mid-size model, and a more instructive number than the DEFAULT_CONTEXT_WINDOW = 1_000_000 that packages/llm/llm-deepseek/src/adapter.ts:132 declares, because a million tokens hides the problem rather than solving it.

Price the envelope with the dsh heuristic. A system prompt of about 6,000 characters — harness identity, a deployment persona, and guidance for eight tools:

systemTokens = ceil(6000 / 4) + 4 = 1504

Eight tool schemas serialising to about 14,000 characters of JSON:

toolsTokens = ceil(14000 / 4) + 4 = 3504

So the envelope is 5,008 tokens before a single word of conversation. Now budget the rest:

The headroom figure is not arbitrary. Decompose it: 8,192 tokens for the model's maximum response, plus roughly 16,000 tokens for one large tool result — a 64KB file read prices at ceil(65536 / 4) + 4 + 4 ≈ 16,392 — plus a small margin for the heuristic being wrong. If your tools can return more than that, your headroom must be larger or your tools must truncate, which is exactly what Module 3's M3.5 result shaping was for.

Now the ratio question. dsh does not express the trigger as "leave 25,488 tokens free"; it expresses it as a fraction of the window, in packages/compaction/compaction-basic/src/config.ts:

const DEFAULT_THRESHOLD_RATIO = 0.8
const DEFAULT_RETAIN_RATIO = 0.16

With thresholdTokens = Math.floor(contextWindow * policy.thresholdRatio), a 128k window triggers pressure handling at 102,400 tokens — leaving 25,600 free, which is where the table's headroom figure comes from. The retainRatio of 0.16 gives retainTokens = 20,480: the amount of recent history compaction must preserve verbatim. The config enforces the relationship that makes this coherent:

if (retainTokens >= thresholdTokens) throw new TargetPressureConfigError(...)

If you retained more than you triggered at, compaction could never make progress — it would summarise nothing and immediately re-trigger. Failing at config load rather than at 2am under pressure is the point.

Ratios travel across models; absolute token counts do not. A policy tuned as "leave 25,000 tokens free" is correct for a 128k model and catastrophic for an 8k one. But ratios are wrong at the extremes in the other direction: 20% of a one-million-token window is 200,000 tokens of headroom, which is far more than any response plus tool result needs, so a pure ratio wastes most of a large window. The honest answer is that neither pure form is right, and dsh lets retainTokens be set absolutely as an alternative to retainRatio — the two are mutually exclusive in config, precisely because expressing the same quantity twice invites contradiction.

The deeper tradeoff is estimation accuracy. Four characters per token is a reasonable average for English prose and a poor one for minified JSON, base64, CJK text, or source code with long identifiers. dsh's own documentation lists meter accuracy as a known limitation: the figure follows a fixed heuristic, not the model's real tokeniser. Running the actual tokeniser would be exact but requires shipping and maintaining per-model vocabularies, and costs real CPU on every measurement. The heuristic is a deliberate trade of accuracy for simplicity, paid for by keeping headroom generous enough to absorb the error.

dsh separates the three budget categories explicitly in a session projection, packages/llm/token-meter/src/breakdown-projection.ts, whose state is exactly:

{ systemTokens, toolsTokens, messageTokens, claim? }

Envelope figures are last-wins per request/header event; the message figure folds incrementally over surface changes. Because it is a projection over the append-only session log rather than a recomputation, it stays O(1) in state size over a session's life and reports the same numbers as measure() at every event boundary.

Claude Code surfaces occupancy to the user directly — a visible percentage and a warning as the window fills — treating the budget as something the human co-manages. OpenAI's Codex CLI has historically leaned on smaller effective histories and more aggressive turn-level resets. The design axis is whether the budget is a user-visible resource or a harness-internal one. Exposing it trains users to intervene before the harness must, at the cost of leaking an implementation detail into the interface; hiding it makes the tool feel simpler until the moment it silently discards something the user cared about.