Harness vs environment
A harness runs the agent. An environment grades it.
That is the whole distinction, and almost every confused conversation about agent reliability comes from collapsing the two. This page draws the boundary precisely, because where you put it determines what survives your next framework migration.
What is the difference between a harness and an environment?
A harness is the software around a model that turns it into an agent, and an environment is the software around a task that defines what the task is and whether it was done correctly. The harness is model-facing. The environment is work-facing.
LangChain's Vivek Trivedy defines the harness by exclusion in The Anatomy of an Agent Harness: "A harness is every piece of code, configuration, and execution logic that isn't the model itself." That is a useful definition for the harness and a bad one for the system as a whole, because it sweeps the task definition and the grading into the same bucket as the retry loop. They are not the same bucket. One of them changes every quarter.
An environment holds three things a harness does not: a definition of the task and its steps, the state those steps operate on, and the checks that decide whether each step passed. In classic reinforcement learning this is uncontroversial, because the environment is a game and the score comes from the game. In LLM agent work the environment is often missing entirely, and the harness ends up improvising all three.
Where exactly is the boundary?
The harness decides what to do next. The environment decides what happened and whether it was right.
Concretely, the harness owns the model call, the prompt and context assembly, tool selection, retry and stop logic, compaction, subagent orchestration, and session memory. The environment owns the step map, the state store, the action validation, the per-step verifier, the checkpoint, the reset, and the run record. When an agent takes an action, the harness produces the action and the environment applies it, validates it, grades it, and writes it down.
The test for whether you have drawn the line correctly: swap the harness for a different one and ask whether your definition of correct work went with it. If it did, the line is in the wrong place.
Harness vs environment: comparison table
| Harness | Environment | |
|---|---|---|
| Job | Get the model to act well | Define the work and grade it |
| Faces | The model | The business process |
| Owns | Context, tools, retries, stop conditions, memory, orchestration | Step map, state, action validation, checks, checkpoints, run records |
| Answers | "What should the agent do next?" | "What happened, and was it correct?" |
| Unit of change | A prompt edit, a tool, a middleware hook | A step, a check, a tolerance |
| Rewritten when | The model or framework changes, roughly quarterly | The actual process changes, rarely |
| Produces | Actions | Grades and a record |
| Portable across | Tasks | Harnesses and models |
| Failure looks like | The agent flails, loops, or stalls | Nothing catches a wrong answer |
Why does the distinction matter?
Three reasons, and the first two are measurement problems.
Benchmarks measure the harness as much as the model. Scaffold Effects on GAIA: A Controlled Comparison (June 2026) is a controlled study of scaffold choice on a fixed benchmark. Its largest within-model spread was Claude Opus at Level 2 on the robust slice, where "scaffold choice alone moves measured accuracy by as much as 28 percentage points," and its conclusion is that "single-scaffold capability numbers are scaffold-conditional estimates." Harness-Bench reaches the same place from the other direction across 5,194 trajectories: "agent capability should be reported at the model-harness configuration level rather than attributed to the base model alone." A third paper, Stop Comparing LLM Agents Without Disclosing the Harness, reports that harness-induced variance can exceed model-induced variance badly enough to produce "cases of model ranking reversal." A number reported against an environment is only interpretable if the harness is disclosed.
Verification lives on the environment side, so it survives harness churn. If your checks are implemented as prompts inside your agent loop, they are coupled to a framework with a six-month half-life. If they are implemented as verifiers the environment applies to step outputs, you can replace the entire harness on a Tuesday and your pass rate is still comparable to last month's. This is the argument for treating the environment as the durable asset and the harness as consumable.
Training needs the environment, not the harness. Reinforcement learning with verifiable rewards needs a reward that comes from a check rather than from a preference model, plus the ability to reset a task and attempt it many times. Those are environment capabilities. A harness cannot supply them, and an agent stack with no environment underneath it has logs rather than training data.
Can one environment serve several harnesses?
Yes, and that is the point of the separation. If the environment exposes a stable observation surface, a validated action surface, checkpoints, reset, verifier hooks, and audit emission, then any harness that speaks that interface can run in it. You can put a frontier model with a commercial coding agent in one week and your own fine-tuned open model in another, and compare them on identical tasks with identical grading. The interface is specified in what an environment must expose to a harness.
Running it the other way around does not work. One harness across many environments is normal and fine, but it gives you no comparability, because every environment grades differently.
Does the harness or the environment own verification?
The environment, always. A harness can carry its own checks, and good harnesses do, but a check the agent's own runtime owns is a check the agent can be tuned to satisfy. Lilian Weng makes the same point about optimization loops in Harness Engineering for Self-Improvement: held-out tests, trace audits, and human review have to sit outside the loop being optimized. Move grading to the environment and that separation is structural rather than a matter of discipline.
Pebble builds the environment side. A machine does the steps on your systems, a check you approved grades each one, and the graded runs become training data for a model you own. Failed runs stop and resume at the failed step, and the run records are the audit trail and the training set at the same time. For the vocabulary, see the glossary, for the harness itself see agent harness, and for how the environment side gets built see environment engineering.
If you have a harness that works in a demo and cannot be trusted in production, write to [email protected] for a scoping call.