How to verify AI agent work

How do you verify AI agent work?

You verify AI agent work by breaking the work into numbered steps, attaching a pass condition to each step that was approved before the run, grading every real run against those conditions, and stopping the run at the first step that fails. Anything less than per-step grading on real runs is testing, and testing answers a different question.

Why is testing not verification?

Testing is not verification because a test runs before deployment on cases you chose, while verification grades the run that actually happened on the input that actually arrived. A passing test suite tells you the agent handled your examples. It says nothing about the run that failed at 2am on an invoice in a currency you did not think about.

Apica's writeup of this is the sharpest version: "An AI agent returns HTTP 200. The JSON is valid. The response mentions the right company name, references the correct policy number, and sounds confident. Every status check passes. Every dashboard shows green. The agent hallucinated its findings, skipped a required tool call, and hid the failure behind a well-formatted answer" (Your AI Agent Passed Every Test. It Was Still Wrong). Apica's answer is synthetic monitoring that checks whether the agent completed its work rather than whether it responded, the right instinct applied from the outside in.

The gap it points at is structural. Observability tells you what the agent did. Format validation tells you the output is well-formed. Neither tells you the output is right, because rightness is a property of your process, not of the response. Somebody has to write down what "right" means at each step and approve it.

What is an AI agent verifier?

An AI agent verifier is the component that decides whether the agent's output at a given step meets the standard, and it can be code, a trained classifier, a judge model, or a human. The word covers all four, which is why "we have a verifier" is not information. What matters is which kind and what its measured error rate is.

Which verification methods actually work?

Ranked from most trustworthy to least, with what each one costs you.

1. Environment-state assertions and computed checks. Code reads the state of the systems the agent touched and decides. Do the two ledgers agree, does the record exist with the expected values, does the sum of line items equal the invoice total. These are correct by construction: they do not drift, and they return the same verdict no matter how strange the agent's behavior gets. The cost is engineering time and access to real system state. Push as many checks as possible into this tier. More of a business process is computable than people assume once it is decomposed into small enough steps.

2. Trained classifiers with holdout scores. A small model trained to make one narrow decision for one check, scored against held-out examples before it is trusted. Not a general-purpose grader. A single-purpose classifier with a number attached. The number is the point: if you cannot state the holdout accuracy, you do not have a verifier, you have a second model whose errors correlate with the first one's.

3. Judges with measured human agreement. Where the pass condition is genuinely a judgment call, have the operators who own the process annotate a few hundred pass/fail examples, train a judge on them, and report the judge's agreement with those humans on held-outs. Where agreement is too low, the human stays in the check. What makes this tier work is the measurement, not the model.

4. LLM-as-judge, unaudited. A frontier model with a prompt asking "is this correct". The most common thing shipped and the weakest thing on this list. The paper that established the method also established its holes: Zheng et al. found GPT-4 judges reach over 80% agreement with human preferences, matching human-to-human agreement, while documenting "position, verbosity, and self-enhancement biases, as well as limited reasoning ability". Follow-on work quantifies position bias and self-preference bias. An unaudited judge is not worthless, it is unmeasured, which means you cannot report a reliability number off it and will not notice when it drifts.

5. Self-verification. The agent checks its own work. Treat this as a floor. The assumption that checking is easier than doing is a complexity argument that does not transfer to systems working by approximate retrieval, and the empirical result is blunt: across Game of 24, graph coloring, and STRIPS planning, Stechly, Valmeekam, and Kambhampati report "significant performance collapse with self-critique" and "significant performance gains with sound external verification". A model does not catch the error classes it reliably makes. Fine as a cheap first pass, useless as the thing you report to an auditor.

The ordering is a preference, not a taxonomy. The engineering job is moving each check up the list until it stops moving. The choice between tiers 1 and 4 for a given step is worked through in verifier vs LLM judge, and the mechanics of building the check itself are in grader design. Terms are defined in the glossary, and why programmatic checks beat learned reward signals is covered in reinforcement learning with verifiable rewards.

How do you verify an agent in production?

You verify an agent in production with the run-record pattern: number the steps, attach a check to each step, grade every run, stop at the first failure, and keep the graded record. Four parts.

Numbered steps. The process is decomposed into steps with defined inputs and outputs before any agent runs it. This is the artifact that turns "the agent got it wrong" into "step 4 failed". Without it, every failure investigation is a transcript read.

Per-step checks. Each step has its own pass condition, from the tiers above, approved by the person who owns the process. Approval matters as much as the check: a condition you did not sign off on is a vendor's opinion about your work. What has to be true before a condition counts as approved is set out in approved checks.

Stop and resume. When a step fails its check, the run halts at that step and resumes from there once the problem is fixed, rather than restarting from zero. This is what makes long processes economical. One bad extraction at step 3 does not throw away steps 1 and 2.

The run record. Every run leaves a record: each numbered step, the check applied, the verdict. It is the audit trail when someone asks what happened on a specific transaction, and accumulated across runs it is a training set. The fields that record has to carry, and why an entitlement log is not a substitute, are in verified agent audit trail. Reliability then becomes a percentage read off run history rather than a claim: how often this step passed its check across the last N runs. The line between the code that drives the agent and the environment that grades it is covered in agent harness.

What does a verified run look like in practice?

Take invoice reconciliation, which decomposes into steps: pull the invoice, extract header and line items, locate the purchase order, locate the goods receipt, match them, flag exceptions, post to the ledger.

The three-way match is a computed check and a good example of what tier 1 buys you. It is code, not judgment: for each line, does the invoice quantity equal the received quantity, does the invoice unit price equal the PO unit price, and does the invoice total equal the sum of its lines within the tolerance the controller approved. All three conditions read state from the ERP and the receiving system. The verdict is deterministic. It does not care whether the agent got the values by OCR, by API, or by guessing, and it cannot be talked into a pass by a confidently worded summary.

Not every step is that clean. "Is this exception note clear enough for the AP clerk to act on" is not computable. That becomes a judged check trained on a few hundred notes the AP team annotated as actionable or not, with the judge's agreement reported on held-outs, and a human reviewing anything the judge is unreliable on.

What can this still not catch?

Plenty. A check only catches what it encodes, learned checks drift, and optimization pressure finds holes in graders. That accounting is in the limits of AI agent verification. If you landed here looking for the other meaning of the term, the identity-verification market is covered in agent verification.

Where do you start?

Start with one process and find its computed checks, because that is where honest reliability numbers come from and it tells you quickly whether the rest is tractable. If you want a second opinion on whether your process decomposes into checkable steps, book a scoping call at [email protected]. If an environment is the wrong tool for the job, we say so on the call.