What an environment must expose to a harness

An agent harness can only do what the environment lets it do. If the environment is a pile of API calls with no state model, the harness invents one, badly, and the resulting agent cannot be graded, resumed, or trained on. This page specifies the capability surface an environment must expose for any harness to run agents inside it: seven capabilities, each with a contract, an example, and what it buys in training and in production. Written by Pebble, which builds environments around real business processes. Nothing here is proprietary, so treat it as a spec to hold your own environment against, or to hold us against.

The capability matrix

The last two rows are one object seen twice. A run record is the audit trail and the training set at once, which is the argument for building the environment before the agent.

Observation surface: what can the agent see?

The observation surface is what the environment will show an agent at a given step, in a stable format, with nothing hidden behind a side channel. The contract: two agents at the same step of the same task see the same thing, and the observation is complete enough to act on without out-of-band context.

In invoice reconciliation, the observation at the matching step is the invoice as extracted fields plus a reference to the source image, the candidate purchase orders, the receiving records, and the vendor's payment terms. Not a chat transcript summarizing those things. If the harness reconstructs state from prior messages, every run diverges a little differently.

For training, an unstable observation surface makes rollouts non-comparable and credit assignment noisy. For production, if you cannot show what the agent saw, you cannot defend what it did.

Action surface: what is the agent allowed to do?

The action surface is the finite, typed set of state changes the environment accepts, validated individually, with no path to mutate state outside it. Every action either applies or is rejected with a reason. There is no third outcome.

In safety-report triage, the action surface at the classification step is small: assign a severity, attach a hazard code, mark the report a duplicate of an existing case, request a missing field from the submitter, or escalate to a human reviewer. It does not include "write arbitrary SQL."

For training, a bounded action space makes the problem learnable. For production, it states what the agent can do to your systems before an incident rather than after one. Auditing Agent Harness Safety found most violations it caught were resource-access and cross-agent information-transfer failures, which are action-surface problems rather than model-intent problems.

State introspection: can the agent ask questions between actions?

State introspection is read-only querying that does not advance the run. The contract: it is free, side-effect-free, and logged separately from actions so it never contaminates credit assignment.

The triage agent should be able to ask "how many open cases already reference this asset ID" without that counting as a triage decision. In invoice reconciliation it is "show me the last six invoices from this vendor" before deciding whether a price variance is normal.

For training, introspection lets a policy be scored on how well it gathers evidence rather than only on its final answer. For production, it is the difference between a postmortem that reads a few queries and one that replays a whole run.

Checkpointing: is a completed step durable?

Checkpointing is the environment committing a step's outcome as a named, durable point a run can restart from. The contract: a checkpoint stores enough state to resume without re-executing anything before it. In invoice reconciliation, extraction, PO matching, receipt matching, GL coding, and approval routing are separate checkpoints. A failure at GL coding does not re-extract the invoice. Failed runs stop at the failed step and resume there.

For training, checkpoints make per-step credit assignment possible instead of one scalar for a long trajectory. For production, they turn "the run failed" into "step 4 failed," and a fix costs one step of compute rather than the whole run. Anthropic's work on long-running agents reaches the same requirement from the harness side, carrying state across context windows in an init script and a progress file. A checkpointing environment means the harness need not improvise that.

Reset and replay: can you run the same task twice?

Reset returns the environment to a defined starting state. Replay re-executes a recorded action sequence against it and produces the same outcome. The contract: determinism where it is possible, and explicit declaration of the parts that are not, such as an external API whose responses must be recorded and played back.

For triage, reset means loading the same report against the same backdrop of open cases, so two policies face an identical situation. Replay means taking last Tuesday's disputed decision and running it again, action for action.

For training, reset is a hard prerequisite. RL needs many attempts at one task, and without reset you have a log rather than an environment. For production, replay is how you reproduce an incident instead of arguing about it. See the glossary for how reset relates to rollouts and tasksets.

Verifier hooks: how does a step get graded?

A verifier hook is a per-step grader the environment owns and applies to the step's output, returning a pass or fail plus a reason. The harness takes the step, the environment grades it, and grading never lives inside the policy being graded.

Pebble uses three check types, in strict order of preference:

1. Computed. Code decides. Do the invoice lines sum to the header total, does the total match the PO within tolerance, does the reported incident count match the sensor count. This is the gold standard, and the goal is to push as many checks into it as possible.
2. Classified. A small model trained for that one check, scored against held-out examples before it is trusted. Catching an illegible field on a scanned invoice is a classified check.
3. Judged. For outcomes needing human taste, such as whether a triage narrative adequately describes the hazard. Operators annotate examples, a judge is trained on them, and judge-human agreement is reported on held-outs. Where agreement is too low, a human stays in the check.

For training, these hooks are the reward. That is the content of reinforcement learning with verifiable rewards: the signal comes from a check rather than a preference model, so the agent cannot win by being persuasive. For production, the same hooks are your pass rate, read off run history rather than asserted in a slide.

Audit emission: what does the environment write down?

Audit emission is the environment's obligation to record, for every step, the observation, the action, the verifier result, the reason, and the timestamp, in an open format with no vendor attached. The contract: the record is written by the environment rather than by the agent, because an agent that writes its own audit log is grading its own homework.

For training, this is the dataset: step number, observation, action, verdict. Passing runs weigh more, and when a stronger open base model ships, the same records train its replacement. For production, this is what compliance asks for and what a postmortem starts from.

Why specify it this way

Because harnesses churn and environments do not. If your verification, state, and audit obligations live in the harness, you re-earn trust with every swap. If they live in the environment, the swap is an afternoon. That boundary is the subject of harness vs environment.

To get this surface built around one of your processes, write to [email protected] for a scoping call.