Three kinds of code I stopped letting an agent write

This is a negative result, and the honest framing matters: in none of these cases was the generated code wrong.

It was plausible, idiomatic and well-structured. The problem was that I could not tell whether it was right without doing most of the work myself.

The case this post is about#

One list of protected directories, typed by hand and reviewed by eye:

LITERAL_PROTECTED=(
  "workspaces/org/first-app/idea-layer/screens"
  "workspaces/org/first-app/idea-layer/flows"
  "workspaces/org/second-app/idea-layer/screens"
)

Every entry on it is correct. That is what a careful review of this diff establishes, and it is the whole problem: the filesystem had twelve such directories, so nine were unprotected and none of the nine appears anywhere in the change.

The generated version and the correct version differ only against a tree that neither the author nor the reviewer looked at.

This post follows that one list, because the output was not bad — it was plausible, internally consistent, and exactly what was asked for, and reviewing it could not have found the defect.

The symptom#

The economics of generated code are usually stated as writing time saved. That is the smaller term. The real question is verification cost, and it varies enormously by category in a way that writing time does not.

For most code, verification is cheap because correctness is visible locally. A function that transforms a list either handles the empty case or does not, and you can see which by reading it. For some code, correctness lives somewhere the source does not show you — in a filesystem layout, in a third-party service's behaviour, in a build that runs elsewhere — and then reading the diff tells you almost nothing.

The shape of the failure is specific: you accept a change you cannot fully evaluate, because it looks exactly like one you could. Fluent output and correct output are indistinguishable at a glance, and the glance is what a saved hour buys you.

The first: guards whose value is what they refuse#

A protective hook accumulated seven recorded bypasses over its lifetime. Each was patched by widening what it inspected, and each patch looked correct in review.

The reason they kept slipping is that a guard's correctness is defined entirely by the inputs it rejects — which are, by construction, not present in the diff. You can read a matcher carefully, agree with every clause, and learn nothing about the one shape it misses, because that shape is not written anywhere.

I did not stop generating guards. I stopped accepting them without a red case:

nine shapes that must stay refused   → written first, run before the change
one shape that must now pass         → the false positive being fixed

The change to that hook was the first narrowing it had ever had, against seven widenings. I would not have dared it on a reading — I dared it because a set of known-bad inputs was still being refused afterwards, which is a claim a machine settles.

The second: anything whose truth is on disk#

The second category is code that asserts something about the filesystem — a path, a layout, a glob, a build output location.

A list of protected directories in that same hook had been typed by hand, and was nine entries behind reality. Every entry on it was correct. The nine missing ones were unprotected, and no amount of reading the list would reveal that, because the list is internally consistent and the filesystem is not in the diff.

This is the category where generated code is most seductive, because it produces exactly what you asked for: a plausible list of directories, a sensible-looking glob, a path that reads correctly. The generated answer and the correct answer differ only against a filesystem neither of you looked at.

What I do now is refuse the literal and require the derivation:

_DERIVED="$(ls -d "${ROOT}"/workspaces/*/*/idea-layer/*/ 2>/dev/null | ...)"

Not because it is more elegant. Because it cannot be stale, so verifying it means checking one expression rather than auditing a list against a tree.

The derivation and the literal can be compared, which is the only review of a literal list that actually settles anything:

# The literal, as typed by hand and reviewed by eye.
LITERAL_PROTECTED=(
  "workspaces/org/first-app/idea-layer/screens"
  "workspaces/org/first-app/idea-layer/flows"
  "workspaces/org/second-app/idea-layer/screens"
)

audit_literal() {
  local root="$1" missing=0 stale=0

  # The DERIVED answer. One expression, cannot be behind the filesystem.
  local -a derived
  while IFS= read -r d; do derived+=("${d#"$root"/}"); done < <(
    list_dirs "$root"/workspaces/*/*/idea-layer/*/ | strip_trailing_slash
  )

  local d l
  for d in "${derived[@]}"; do
    contains "$d" "${LITERAL_PROTECTED[@]}" && continue
    echo "UNPROTECTED: $d exists on disk and is not in the literal list" >&2
    missing=$((missing + 1))
  done
  for l in "${LITERAL_PROTECTED[@]}"; do
    contains "$l" "${derived[@]}" && continue
    echo "STALE: $l is in the literal list and does not exist" >&2
    stale=$((stale + 1))
  done

  echo "literal ${#LITERAL_PROTECTED[@]} entries · derived ${#derived[@]} · $missing unprotected · $stale stale"
  [ "$missing" -eq 0 ] && [ "$stale" -eq 0 ]
}

Run against the tree the list claimed to describe:

$ audit_literal "$ROOT"
UNPROTECTED: workspaces/org/first-app/idea-layer/exports exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/first-app/idea-layer/journeys exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/first-app/idea-layer/mockups exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/first-app/idea-layer/preview exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/first-app/idea-layer/state exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/first-app/idea-layer/testers exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/second-app/idea-layer/flows exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/second-app/idea-layer/state exists on disk and is not in the literal list
UNPROTECTED: workspaces/org/third-app/idea-layer/screens exists on disk and is not in the literal list
literal 3 entries · derived 12 · 9 unprotected · 0 stale

Zero stale and nine unprotected, which is exactly the shape that makes a hand-written list dangerous. Every entry on it was correct — reading the list carefully confirms that, and confirming it is what a review of this diff produces. The nine the list does not mention are not in the diff at all, and no amount of attention to the three that are there can surface them.

The third: ordering across steps nobody can see together#

The third is sequencing between components that no single file contains.

A site build writes to one directory, a step turns that into another, and a quality gate audits the result. The gate reported a confident pass on 113 files when 112 had just been written — it was reading the previous build, because it ran before the rebuild.

Every one of those three components is correct in isolation. The defect is entirely in the relationship, and there is no file you can open that shows the relationship. A reviewer reading the gate sees a gate. A reviewer reading the build sees a build.

I have not solved this one, and I want to be straight about that. What helps a little is refusing to accept cross-step ordering as a diff at all — the change has to come with a run whose output shows the new order, because the only representation of ordering that can be checked is an execution.

What the three have in common#

In all three, the information needed to verify the change is not in the change. It is in the set of inputs the guard will see, the state of a filesystem, or the interleaving of steps that live in different files.

That is the predictor, and it is more useful than any category list, because it tells you what to do rather than what to avoid. When the truth is elsewhere, generation is still fine — what changes is that acceptance requires an execution rather than a reading. A red case that goes red. A derivation instead of a literal. A run whose output shows the ordering.

The corollary is the part I find genuinely useful day to day: if you cannot construct that execution cheaply, that is the signal to write the thing yourself. Not because the agent would get it wrong, but because you are about to accept something on the strength of it looking right, and this is the category where looking right is uncorrelated with being right.

Where I was wrong about this#

For balance: I held a fourth category on this list for a while, and I was wrong about it.

I believed anything touching money or credentials should be hand-written on principle. What actually went wrong in that area was not generated code — it was a command I chose, to list field names in a properties file, which printed a secret because the file was malformed. The authorship was never the risk. The missing allowlist was.

Moving that category off the list and replacing it with a default-deny on the surface itself was strictly better: it protects against every author, including the careful one, including me.

That is worth generalising. "Do not let the agent write this" is a weak control, because it depends on remembering to apply it. "This surface refuses anything that is not on a small allowlist" holds regardless of who is typing.

What this generalizes to#

Before accepting generated code, ask where the information that would prove it correct actually lives. If it is in the diff, read the diff — that is the common case and generation is a clear win.

If it is somewhere else — in rejected inputs, on a filesystem, in the ordering between steps — then require an execution instead of a reading, and treat the cost of building that execution as the real price of the change. When that price is higher than writing it yourself, write it yourself. And where a whole surface is dangerous regardless of author, put the guard on the surface rather than a rule on the author.