Asking AI to check its own work does not work. Ask it to disprove it.

An agent finished a task and reported success. I asked a second agent to check it. The second agent agreed.

Both were wrong, and the checking step had made me more confident rather than less.

TL;DR#

• A verifier told to confirm will confirm — the instruction decides the verdict more than the evidence does.
• Change the instruction from CONFIRM to REFUTE and the job becomes generative rather than evaluative: it must produce a counter-example, not an opinion.
• When to use several refuters with different lenses, and when one is enough.

The case this post is about#

One scroll loop, asked to enumerate a list, and one second agent asked to check it:

let prev = -1
for (let i = 0; i < 60; i++) {
  const n = (await rows(page)).length
  if (n === prev) break          // "nothing new loaded, we're done"
  prev = n
  await page.mouse.wheel(0, 20_000)
}

That loop returned 17 items. The real count was 100 — the break fires on any frame where the fetch had not landed yet, which on a slow response is the second iteration.

The first agent reported success. The second agent, asked to check it, agreed. Both numbers were wrong and the checking step had raised my confidence rather than lowering it, which is strictly worse than not having checked.

The symptom#

A verification pass that never fails is not a verification pass. Mine returned green on work that was visibly broken, repeatedly, and the failures were not subtle:

• A delete step reported deleted: true. The post was still in the drafts list.
• An image-embedding step exited 0. The image was never embedded, because the function had thrown.
• A list of published posts returned 17 items. The real number was 100.

In each case something downstream had "verified" the step. In each case the verification was a restatement of the claim.

Why the obvious fix failed#

The obvious fix is to ask a second model to review the first. It is the right shape and the wrong prompt, because "check whether this is correct" is answered by pattern-matching plausibility, and a plausible wrong answer is exactly what you are trying to catch.

Worse, the second agent sees the first agent's reasoning. That reasoning is fluent and internally consistent — it was generated to be. Agreement is the cheapest available response and nothing in the instruction penalises it.

I watched this happen with a scroll loop. The code collected rows from a lazily-loaded list:

let prev = -1
for (let i = 0; i < 60; i++) {
  const n = (await rows(page)).length
  if (n === prev) break          // "nothing new loaded, we're done"
  prev = n
  await page.mouse.wheel(0, 20_000)
  await page.waitForTimeout(700)
}

Asked "does this correctly read the whole list?", a reviewer says yes. It scrolls, it re-counts, it stops when the count stabilises. Every individual claim is true.

It stopped at 25 of 100 items. The list was paginated at 25, and one 700ms wait was not enough for the next page, so the first quiet round was treated as the end. Confirmation-shaped review cannot see this, because the bug is not in what the code says — it is in what the code assumes.

The actual mechanism#

Change the instruction from confirm to refute, and the model's job becomes generative rather than evaluative. It is no longer asked whether something looks right; it is asked to produce a counterexample. Producing a counterexample is a task it can actually do.

The prompt that works is close to:

Try to prove this wrong. Construct an input or a state where it fails. Default to refuted if you cannot establish otherwise.

That last clause matters more than it looks. Without it, "I could not find a failure" collapses back into "it is correct" — and those are different statements. One is evidence of absence, the other is absence of evidence.

Against the scroll loop, a refuter reaches for the obvious adversarial case immediately: what if the next page has not arrived yet? And the loop is dead, in one sentence, with no knowledge of pagination required.

The fix#

Three things made refutation actually usable rather than just noisier.

Ask several, take a majority. A single refuter has the opposite failure mode — it manufactures objections, because it was asked for objections. Three independent attempts with a majority rule cancels most of that:

const votes = await Promise.all(
  Array.from({ length: 3 }, () =>
    agent(`Try to refute: ${claim}. Default to refuted=true if uncertain.`, { schema: VERDICT })),
)
const survives = votes.filter((v) => !v.refuted).length >= 2

Give each one a different lens. Three identical refuters produce three correlated answers. Three asked for correctness, does-it-reproduce, and what-happens-at-the-boundary produce three different kinds of attack. Diversity does work that redundancy cannot.

Make the refutation checkable, not rhetorical. A verdict of "this may fail under load" is unfalsifiable and therefore worthless. A verdict of "with 100 items and a 25-item page size it returns 25" is a test. Requiring concrete inputs and an expected wrong output is what separates a refutation from an opinion.

How we proved it#

The scroll loop is the clean case, because the fix produced a number.

Refutation named the failure — page size, not end of list. The repair was to require several consecutive quiet rounds and to cross-check the total against the count the page itself printed. Two independent reads then agreed exactly:

counts: {"published":{"declared":100,"scraped":100},"draft":{"declared":17,"scraped":17}}

Before: 25 scraped, reported as complete. After: 100 of a declared 100, twice.

The second case is sharper, because refutation found a bug in the verifier. A check confirmed that appended text had landed by searching the page for the string it had typed. It reported the text missing. The text was plainly there.

The editor had applied smart typography: web — the was stored as web—the. The comparison was byte-for-byte, so it failed on text that was present. A confirming reviewer reads that code and sees a reasonable check. A refuter asks what the editor does to the string between typing and reading, which is the whole bug.

That one is worth dwelling on. The verification step was the defect, and the only thing that could find it was an attack aimed at the verifier rather than at the code under test.

What I cannot give you is a rate. I know refutation caught these two, and I have no count of how often it fires against how many claims — the denominator would need every claim ever passed through it, and I only kept the ones where the verdict changed something. So the case for this is two named incidents and a mechanism, not a measured hit rate, and anybody adopting it on the strength of this post should expect to discover their own.

And one cost that is not obvious. Three refuters per claim is three calls where a confirmer was one, and the diversity requirement means they cannot share a prompt — so the expensive version is the one that works. On a long verification pass that is a real multiplier, and the honest place to spend it is the claims whose failure is expensive rather than every claim. Deciding which those are is a judgement nothing here automates.

Where it does not work#

Refuters are wrong often, and the ways they are wrong have a shape worth knowing before you trust the output.

They invent failure modes that cannot occur. Asked to attack a function that takes an enum, a refuter will propose passing it a string — true of the type system it imagined, not the one in front of it. They also refuse things for being unlovely: "this would be clearer as a switch" is not a counterexample, and a refuter that returns it has answered a different question. Requiring concrete inputs and an expected wrong output filters most of both, because neither survives being asked to name the input.

There is a third failure that the input requirement does not filter, and it is the one to watch for: a refuter that attacks a real weakness in something adjacent. Asked to refute the scroll loop, one returned a genuine problem with the selector the loop used to count rows — correct, worth fixing, and not a refutation of the claim under test. The verdict came back refuted: true with a finding that left the original claim exactly as uncertain as before. Scoping the question to one claim and rejecting findings that do not bear on it is manual work, and I have not found a way to make the instruction do it.

The deeper limit is that refutation is only as good as the claim it is given. "The scroll loop works" is attackable because it is specific enough to be wrong. "The pipeline is reliable" is not a claim a refuter can engage with, and asking it to try produces exactly the unfalsifiable mush the input requirement exists to reject. So the discipline starts before the refuter: if you cannot state the claim as something that could be false, the verification step has nothing to do and dressing it up as an adversarial check just spends three calls agreeing.

That is the part I would keep if I had to drop everything else here. The instruction to refute is cheap to adopt and it does not rescue a vague claim — it only pays on a claim specific enough to have a counterexample, which means most of the value arrives before any agent is called, in the work of saying precisely what you believe happened.

The harder limit is that refutation only attacks the claim you make. A step that reports deleted: true gets checked for whether the delete worked. Nobody asks whether the thing deleted was the right one — that is a different claim, and it goes unexamined because it was never stated. Every case I hit shared this: the bug lived in an assumption too obvious to write down, and refutation found it only once the assumption was promoted into a claim someone could attack.

So the practical sequence is to state the assumption first, then attack it. "This deletes the post" is weak. "This deletes exactly the post whose id came from the ledger, and no other" is a claim with an attack surface — and the second version is the one that would have caught a delete routine matching by title.

Cost is the other constraint, and it is not small: three verdicts per claim, each a model call, on every claim you care about. That is roughly four times the price of a single confirming review. It pays where a wrong "verified" is expensive — anything irreversible, anything that writes to a third party, anything where the failure is silent. It does not pay on a typo, and running it everywhere mostly teaches people to skim the verdicts, which is worse than not having them.

What this generalizes to#

Confirmation is the default because it is cheap and it feels like diligence. The move is not more review; it is review with the burden of proof inverted — ask for a counterexample, require it to be concrete, and treat "I could not find one" as the weak evidence it actually is.

---

Related#

• A canary that cannot fail proves nothing — A check that agrees with you because it structurally cannot disagree
• A verification step that can pass while the bug is present is not a verification step — The verification step that passes while the bug is present

Keep reading#

One post a week on shipping software with AI agents — what broke, what the fix was, and the evidence. No digests, no course pitch. Subscribe.