Agent harness engineering: the discipline

Agent harness engineering is the practice of designing and maintaining the software around a model so the model produces correct work: the tool surface it acts through, the context it sees, the checks it gets back, the retry and stop rules, and the memory that survives a context window. It is the layer once called scaffolding, now a discipline with its own failure modes, benchmarks, and research literature. Mitchell Hashimoto coined the name in My AI Adoption Journey on February 5, 2026, defining it as the idea that "anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again." For the noun itself and the contested history of who named it, see agent harness.

Why is harness engineering a superset of prompt engineering and context engineering?

Because a prompt is one input to one model call, context is the whole input to one model call, and a harness governs every call in a multi-step run plus everything between them. Wikipedia's framing is that harness engineering is "a broader layer than prompt engineering, which optimises a single interaction," distinguished by having to design recovery mechanisms, since the wrapped component is non-deterministic.

Recovery is what makes it a different job. Harness engineering answers what happens on the eleventh attempt, when the tool returns a 500, when the context is full, and when the model is confidently wrong. None of that is answerable inside one call.

What does a harness engineer actually decide?

Five decisions, mostly about constraint rather than capability.

Tool surface. Which tools exist, how they are described, and what they refuse. Tool descriptions are prompt surface that nobody treats as prompt surface. The Interplay of Harness Design and Post-Training in LLM Agents (June 2026) makes tool exposure a controllable variable and finds a minimally designed harness produces "a drastic performance drop under stronger tool environment shifts."

Context management. What enters the window, what gets compacted, and what gets written to a file instead of carried in the transcript. The strongest published result here is architectural: Harness-1 trains a 20B search agent inside a harness that externalizes candidate pools, evidence, and verification records, and lands 11.4 points of curated recall ahead of the next strongest open search subagent.

Retry and stop logic. How many attempts, on what signal, with what backoff, and the harder question of when to stop and ask a human. A harness with no stop condition is a way to spend money.

Memory. What persists across sessions and in what form. Anthropic's Effective harnesses for long-running agents (November 2025) is the clearest worked pattern: an initializer agent writes an init script and a feature list, then a coding agent is invoked repeatedly, each session leaving a progress note and a commit.

Verification hooks. Where checks attach and what they check. Birgitta Böckeler's Harness engineering for coding agent users splits these into guides, which steer before the agent acts, and sensors, which observe the result and let it self-correct, each either computational, like a linter, or inferential, like a model asked to judge. Her follow-up on sensors is the practical treatment.

Why does harness quality dominate outcomes?

Because the same model scores very differently depending on what wraps it, by enough to reverse rankings. Scaffold Effects on GAIA: A Controlled Comparison (Jason Starace, June 2026) is a controlled study of exactly this. Its pre-registered hypothesis was that scaffold variation produces gaps of at least 10 points, which held, and its largest within-model spread was Claude Opus at Level 2 on the robust slice, where "scaffold choice alone moves measured accuracy by as much as 28 percentage points." The paper concludes that "single-scaffold capability numbers are scaffold-conditional estimates," and that the elicitation gap is not guaranteed to close as models improve.

The corollary is uncomfortable for anyone reading a leaderboard. A published agent score is a joint measurement of a model and a harness, reported as a property of the model.

How is harness engineering different from model training?

Model training changes the weights and harness engineering changes everything else, on very different cycle times. A harness edit ships in an afternoon and can be reverted. A training run cannot. That is why harness work is usually the right first move: you learn how much headroom exists in elicitation before spending on weights. They are not independent, though. Training inside a well-designed harness produces more robust behavior than training inside a bare one, so the harness is a training decision whether or not you treat it as one.

How is harness engineering different from environment engineering?

The harness runs the agent and the environment defines and grades the task, and they should be separate codebases with separate owners. Harness engineering asks how to get the model to act well. Environment engineering asks what the work is, what state it operates on, and what makes a step correct. Verification belongs on the environment side, which is why it survives harness churn. Your harness will be replaced. Your definition of a correctly reconciled invoice will not. See harness vs environment, what an environment must expose to a harness, and environment engineering for the other discipline in full.

Is there an agent harness engineering survey?

Yes, one survey and a cluster of primary papers, all from 2026. In rough reading order:

1. Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems (May 2026). The survey, organized into harness interface, harness mechanisms, and scaling the harness. Start here.
2. Harness-Bench (May 2026). 106 sandboxed tasks and 5,194 trajectories built to isolate the system layer. Conclusion: "agent capability should be reported at the model-harness configuration level rather than attributed to the base model alone."
3. Stop Comparing LLM Agents Without Disclosing the Harness (May 2026). On long-horizon tasks, among models of comparable capability, harness configuration explains more variance than model choice, "including cases of model ranking reversal." Proposes disclosure requirements for agent evaluations.
4. Agentic Harness Engineering (April 2026). Automates harness evolution through component, experience, and decision observability. Terminal-Bench 2 pass@1 rises from 69.7% to 77.0% over ten iterations, above a human-designed Codex-CLI baseline at 71.9%. Gains localize to tools, middleware, and memory rather than system prompts.
5. Self-Harness: Harnesses That Improve Themselves (June 2026). Weakness mining, harness proposal, then validation by regression test, run by the agent on itself. Reports Terminal-Bench 2.0 gains including MiniMax M2.5 from 40.5% to 61.9%. Its premise is that good harness design is model-specific.
6. Auditing Agent Harness Safety (May 2026). HarnessAudit-Bench, 210 tasks across eight domains under ten harness configurations. Task completion and safe execution diverge, and violations accumulate with trajectory length.
7. Natural-Language Agent Harnesses (March 2026) and Harness Handbook (July 2026). Both attack legibility, one by writing harness policy as documents a runtime interprets, one by synthesizing a navigable handbook from a harness codebase.

One non-paper belongs on the list: Lilian Weng's Harness Engineering for Self-Improvement (July 2026), which argues that held-out tests, trace audits, and human review must sit outside any harness optimization loop to prevent reward hacking.

What does the maturity ladder look like?

Five rungs. Most teams are on rung two and believe they are on rung four.

1. Bare API call. One prompt, one response, a human reads it.
2. Loop with tools. A ReAct-style loop, a few tools, a retry on exception. Works in a demo, fails on anything long-horizon.
3. Engineered harness. Deliberate context management, compaction, subagents, permissions, stop conditions, memory across sessions. Where the OpenAI and Anthropic write-ups sit.
4. Harness with sensors. Deterministic checks the agent runs on its own work, plus inferential checks where code cannot decide. Hashimoto's formulation lives here: every mistake becomes a permanent fix in the harness.
5. Verified environment. The task, the state, the per-step checks, and the run record move out of the harness into an environment the harness plugs into. Grading stops being the agent's responsibility, failed runs resume at the failed step, and accumulated run records are a training set.

Rung five changes what you can claim. At rung four you have an agent that usually works. At rung five you have a pass rate, an audit trail, and data. Pebble builds it: a machine does the steps on your systems, a check you approved grades each one, and the graded runs train a model that has practiced your work. The same structure applies outside business processes. Rockie is an environment we maintain for the public good, running experiment loops for academic groups doing HPC research, and the white paper behind it, Scaling Experimentation, covers making the full experiment loop run as software.

If you do harness engineering and keep hitting the ceiling where you cannot prove the agent is right, that ceiling is an environment problem. Write to [email protected] for a scoping call.