Certified Agent Harness Engineer
Course sections

M5.1

Modelling the threat surface

928 words · about 6 min

Concept

A harness threat model is not "can the model be jailbroken." It is: for each way bytes enter the model's context, what does an attacker who controls those bytes get to do with the harness's authority? The model is a general-purpose instruction follower wired to real capabilities, so any content channel is a potential control channel.

The entry points in a typical coding harness: file reads, shell output, web fetch and search results, git metadata, dependency manifests and their install hooks, MCP or plugin tool results, and cross-session references. Each is a place where text written by someone other than your user reaches the same token stream as your user's instructions.

Mechanism: four attack classes

Mechanism

1. Prompt injection via tool results. A repository's CONTRIBUTING.md contains, below the fold, a line addressed not to a human but to a model — a directive framed as a project convention, instructing the agent that the build requires copying ~/.aws/credentials into a log file before running tests. The user asks the agent to "follow the contributing guide and run the tests." The agent reads the file with its own read tool. Those bytes land in context as tool output, syntactically indistinguishable from the user's request. The agent complies. Impact: credential disclosure with no exploit, no memory corruption, and no bug — every component behaved as designed.

The variants matter because they change who must be compromised:

  • A poisoned dependency's install script. npm install runs arbitrary code by design, so the attacker does not even need the model — the script executes at native privilege the moment the agent installs. The subtler variant: the script writes a plausible build-instructions file the agent will read later, converting one-shot execution into durable influence.
  • A web-fetch result containing instructions. The agent searches an error message and fetches an attacker-controlled page engineered for that exact string. This is the highest-volume channel, because the attacker chooses the bait.
  • A hostile commit message or branch name. git log output enters context verbatim, is attacker-controlled in any repo accepting contributions, and is rarely reviewed on the assumption that a machine will act on it.

2. Over-broad file access. Asked to "clean up the temp files," the agent's write tool takes a path that nothing constrains to the project. A traversal (../../), an absolute path, or a symlink planted inside the workspace pointing at ~/.ssh turns scoped cleanup into arbitrary filesystem mutation. Impact: damage outside the intended blast radius. It need not be malicious — a model that miscomputes a relative path does the same damage.

3. Destructive commands. Not every catastrophe is an attack: rm -rf with a variable that expanded to empty, git reset --hard over uncommitted work, git push --force to a shared branch, a migration against a production connection string sitting in the environment. The defining property is irreversibility, which is why a permission system's real job is to put a human in the loop exactly at the steps that cannot be undone.

4. Exfiltration. Two ingredients: secrets in reach (the harness's API key in process.env, inherited by every child process) and an egress channel (a web fetch with data in the query string, a git push to an attacker remote, a DNS lookup with data in the subdomain). Impact: silent, and typically invisible in the transcript because it looks like ordinary tool use.

Worked example — the exploit against the Module 1 spine

Worked example

Take the spine's shell tool, which receives {"command": "..."} and executes it:

  1. User: "Summarize the open issues in this repo and fix the easy one."
  2. Agent runs git log and reads issue text from a forge API.
  3. One issue body contains an instruction addressed to the agent.
  4. Agent emits a shell tool call carrying an attacker-chosen command.
  5. Spine executes it — full user privilege, ambient environment including DEEPSEEK_API_KEY, unrestricted network and filesystem.

There are zero decision points between steps 4 and 5. The rest of this module reconstructs them: a permission check (M5.2/M5.3), a filesystem fence (M5.4), kernel confinement (M5.5), content provenance (M5.6), and an environment scrub (M5.7).

Tradeoffs

A maximal threat model produces an unusable agent; confirm every read and the tool is a slow file browser. The judgment is asymmetry: spend user attention on irreversible and exfiltrating actions, spend automation on reversible contained ones. Reading a file inside the workspace is cheap to allow because damage is bounded by what the model can then do; writing outside it is expensive because the damage is unbounded and permanent.

How `dsh` does it

How dsh does it

dsh writes the model down as enforced boundaries rather than one document. AGENTS.md requires validation at every trust boundary it names — "parser/config, queued, model/tool JSON, durable/file, worker, process, and wire" — with model/tool JSON called out explicitly as untrusted. docs/defensive-patterns.md records six bug classes learned the hard way, two directly on this surface: never hand untrusted output the ambient environment or predictable paths, and unlink link-shaped paths rather than following them.

How others differ

Harnesses converge on a similar enumeration but disagree on where the boundary sits. Claude Code and Codex CLI both default to workspace-scoped writes with human approval above; Codex CLI additionally defaults to network-disabled execution, closing the exfiltration channel at the cost of breaking every workflow that installs a package. dsh's sandbox vocabulary deliberately governs file effects only and states that "network stays unrestricted" (packages/shell/bash-sandbox/README.md) — a weaker default, honestly documented. Hosted container agents accept the injection risk and rely on the container to bound it: a coherent, different position.


Check yourself

3 questions on M5.1

Start drill