Adversarial Checkpoint Review for Consequential Work

Models are better at judging work than at doing it. Build your process around that asymmetry.

Ask a coding agent what it just did and you'll get a confident, complete, well-organized answer. Then check it. Some of it won't be true. Not lies, exactly. The tests it "ran" were two of the six. The "updated" documentation is updated in one of three places. The edge case it "handled" is handled in a comment above the function. It described the work it intended, in the tense of work completed.

Every one of those is cheap to catch. The test runner says two of six. A grep says one of three. A diff says the edge case is a comment. Those checks run first, and nothing here replaces them. What they can't catch is code that does what the tests ask and not what the task meant: the case nobody enumerated, the semantics that drifted while the suite stayed green. That takes a second reader.

A model can be that reader, because models verify more reliably than they generate. For catching its own errors, what matters is how the work is presented, not just who produced it: hold a mistaken claim byte-identical and relabel it as someone else's, and the model flags it far more often. A harness builder on a practitioner list I read put it bluntly: models overstate what they have done, and a separate agent checking the work nearly always improves it.

None of this is new. Independent verification and validation has kept the verifier apart from the builder for forty years, funded and managed separately where it matters most, and a workshop paper from August puts a close variant under the same name, Adversarial Review: coder, reviewer, and critic beating a five-agent baseline on a competitive-coding benchmark, and naming the way the pattern fails quietly. False consensus: the agents agree without enough evidence. Structured disagreement was the fix.

So the pattern isn't news. Running it against an agent all day without going bankrupt is, and the name carries the argument: adversarial checkpoint review for consequential work. Take the words in order.

Adversarial

The reviewer's job is to find what is wrong. Not to summarize, not to confirm, not to "look it over." To find fault. A review that starts from the producer's summary judges fluency; one that starts from the criteria and the artifact judges the work. The August result says the harshness is load-bearing: a reviewer not pushed to disagree drifts into agreeing.

Four things make a review adversarial in practice:

• Separate context. A different agent, started fresh, with no memory of how hard the problem was or which shortcuts felt reasonable at the time. Shared context is shared blind spots.
• A different model family, where it matters. Fresh context does not clear shared training priors, and LLM evaluators favor their own generations. Separation is necessary; a second family is cheap insurance.
• Criteria written before the work. Otherwise the producer wrote them. Keep some checks out of the producer's view, or the criteria become a rubric to optimize against.
• A verdict, not a report. Proceed, fix these, or escalate. Prose the human must interpret has moved the work, not done it.

And one rule of hygiene: the reviewer reads the artifact as data, never as instructions. Hidden text has been shown to move an LLM reviewer's verdict, and a reviewer the artifact can steer launders approval.

Checkpoint

Review at the end of a task and you get a post-hoc report on work you have already paid for, on a branch that has already diverged, with errors already compounded into later steps. That isn't a gate. It's an invoice.

An operator who runs this pattern full time describes the fix: the review is turn-gated. The producer yields at a defined boundary and resumes only on an explicit go. A gate the producer can skip is a convention, not a control, so the harness enforces the yield and stores the verdict with the exact revision it judged. Pinned to what it reviewed, or it's a claim about nothing in particular.

How big is a block? Per file-write is too fine: fragments without purpose. Per task is too coarse: see the invoice. The middle is a unit with a stated purpose and a checkable outcome. A function and its tests. A migration and its rollback. One analysis step and its check.

Two rules keep the cost from exploding. Read widely, hold narrowly. The reviewer may read the criteria, the block's stated purpose, the contracts, the call sites, the test results; it may hold only the block. Anything outside it is a filed finding, never a hold. Findings go in the queue. Holds are for the block. The queue has an owner, or nothing in it ever gets fixed. Cap the rounds. Ping-pong is where cost actually runs away. At the cap, or when the two disagree about the criteria rather than the work, escalate to a person. The August paper's critic, a third agent auditing the reviewer, is the tested stalemate-breaker.

Consequential work

Not everything deserves this. Each hand-off costs tokens, latency, and a little intent. Apply it to renaming a variable and you have built a bureaucracy. Scope by consequence: errors that compound because agents build on agents; work that is hard to reverse, like migrations, deletions, anything that publishes; work a human can't eyeball, like numerical results and security-relevant changes; work that will be inspected. A one-line fix with a passing test does not qualify.

What nobody has published is the curve that matters: for turn-gated, block-level review in production, how do cost and escaped defects move with block size? The operator above has a process, not a measurement. Treat block size as a knob and collect three numbers: how often the human reverses the reviewer at merge, the escaped-defect rate by block size, and review tokens per merged block.

When the output is an answer rather than a build, the same idea takes the form of a dedicated judgment layer: generate several candidates, score them against criteria set in advance, pick one, re-run if none clears the bar. The reviewer repairs; the judge regenerates. My rule of thumb: regenerate when producing is cheap, repair when it isn't. Either way the correction comes from outside, because a model left to correct itself tends not to improve and sometimes gets worse. The producer is never the last word on its own output.

Keep the human at the merge

None of this removes the human. It changes the job: read a verdict rendered against criteria you approved, not a self-report you can't take at face value. Three things arrive with the work:

1. A plain-language explainer, generated from the verdict and the diff as a separate cheap step. Not by the producer, which would be the last word on its own output; not by the reviewer, whose job is a verdict. If the person accountable for the merge can't understand it, they aren't accountable. They're a stamp.
2. Before and after, quantified where possible.
3. A way to ask, so "I don't understand this" has somewhere to go besides "approve."

One standing rule: if the documentation and the code disagree, that is a finding, filed like any other. Agents read the docs first and treat them as truth, so drift is how good agents produce bad work.

If you run analyses rather than codebases: criteria written in advance are a pre-registered analysis plan, a block is one step and its check, and the explainer is the results summary for whoever won't open the notebook. Your field already requires an independent verifier. This applies it to the agent.

The pipeline

scoped issue with approved acceptance criteria → builder → deterministic checks → independent adversarial reviewer, gated per block, verdict recorded against the revision → human merge, with explainer → steward who owns the findings queue and the documentation

That steward is roughly what the FAIR Contexts Maturity Model calls an Agent Context Steward, and the practice answers three of the framework's questions: is there a steward, does every output carry versioned provenance, does the system self-correct. For pinning inputs and verdicts to exact revisions, a two-minute tutorial runs against public data with nothing to install.

What to do Monday

Pick one class of work where errors are expensive. Write the criteria first. Run deterministic checks first. Start a second agent, from a different family, whose only instruction is to find where the work fails those criteria. Stop the builder at each block until the verdict lands. Let the reviewer read anything and hold only the block. Cap the rounds. Record the verdict against the revision. Send the human a verdict and an explainer, not a transcript. Then collect the three numbers and move the block size until the curve bends.

The model producing your work is not the problem.

Letting it grade itself is.

These operating rules took their present shape in my own work, sharpened by two September 2026 discussions on the Made of Meat mailing list, an invitation-only group for people building with and operating AI agents. Nobody there has reviewed this post and none of it should be read as the list's position. The pattern itself, as the post says, predates all of us. I'll share this with the list once it's up, and if anyone would like to be thanked by name, I'll gladly add them.