A command that checks its own contract

A command in this framework promises four things: it never overwrites an existing file, it never contacts a remote service, it caps itself at five items per run, and it refuses rather than inventing a plan when its input is missing.

Those promises are prose in a document. Nothing about an ordinary edit to that document would tell you if one of them stopped being true.

The case this post is about#

One command, /idea-medium-post-generate, which turns planned rows in a manifest into writable post files. Its contract, as written:

Never overwrites.      a slug with an existing post.md is skipped and reported
Never contacts Medium. scaffolding is local; the irreversible create is a sibling command
Capped per run.        default 5
No manifest -> HALT.   the plan is never invented to fill a gap

The first one is the one with teeth. The command's whole job is to write post files, and the file it must not write is the one that already contains somebody's writing.

The symptom#

The first shape is a promise that quietly stops holding. A runtime document is edited for a dozen ordinary reasons — a section moves, a step is renumbered, a paragraph is compressed because the file hit a line cap. Any of those edits can take the test -f ... && SKIP line out, and the diff looks like a tidy-up. The command still runs, still scaffolds, and now overwrites.

The second shape is that the failure is invisible until it is expensive. Scaffolding over an empty slug and scaffolding over a half-written post produce the same output and the same exit code. You find out when the post you spent an hour on is a template again, and at that point the thing you want to know — when did this stop being safe — is somewhere in the document's history.

The third is the one that makes a prose contract actively misleading. The document still says never overwrites. So the behaviour and the documentation disagree, and the documentation is what anybody reads to decide whether the command is safe to run. A promise that is written down and not enforced is worse than no promise, because it is relied upon.

The fourth is specific to commands whose runtime is a document rather than a program: there is no compiler. A function that loses its guard fails a test. A markdown runtime that loses its guard is still a valid markdown runtime. Nothing in the normal toolchain has an opinion.

Why the obvious fix failed#

Write a test for the command. The instinct, and it does not fit the artifact. The command is a document interpreted by a model. There is no function to call, and a test that executes the command end to end is testing a model's compliance on one sample — useful, slow, and not a statement about whether the guarantee is present.

Review the document carefully. This is the thing that was already happening when the guarantee was lost. A twelve-line diff that moves a section and compresses a paragraph is reviewed as a formatting change, and the deleted guard is one line inside it.

Put the guarantees in a central registry and check against that. Better, and it moves the divergence rather than removing it. Now the registry says "never overwrites" and the runtime may or may not implement it, and the check is comparing one document's claim against another document's claim.

A line-count or hash check on the runtime. Tempting because it is trivial, and it fails in both directions. Every legitimate edit breaks it, so it gets regenerated reflexively, and after the third regeneration nobody reads what changed. A check that fires on every edit is a check that means nothing on the edit that matters.

The actual mechanism#

Assert the properties, one assertion per promise, against the runtime document itself — and prove each assertion can fail by deleting the guarantee it covers.

The second half is what makes this more than decoration. An assertion that has never failed is an assertion that might be matching something else entirely. The fixture that removes one guarantee is the only thing that distinguishes a check from a line that always passes.

Two properties of the assertions matter.

They are per-command, not global. A framework-wide check can assert things true of every command — the file exists, the frontmatter parses. It cannot assert that this command never overwrites, because that is not a property commands have in general. The interesting guarantees are all specific, so they live with the command.

Each one names what it is checking, not just whether it passed. The output is PG-1..PG-6 rather than ok, because a check that fails needs to say which promise is gone. This also matters when adding a new assertion: the existing fixture must keep failing on exactly its own class, or an incidental failure masks the real one.

The fix#

Six assertions over the runtime document, each one keyed to a promise in the contract:

#!/usr/bin/env bash
# capsules/idea/idea-medium-post-generate/verify/verify.sh
set -uo pipefail
ROOT="${CLAUDE_PROJECT_DIR:-$(pwd)}"
RUNTIME="$ROOT/layers/idea/commands/idea-medium-post-generate.md"  # <!-- preflight-ignore -->
BRIDGE="$ROOT/.claude/skills/idea-medium-post-generate/SKILL.md"   # <!-- preflight-ignore -->
ROUTER="$ROOT/capsules/idea/idea-medium-post/CAPSULE.md"           # <!-- preflight-ignore -->
NAME="idea-medium-post-generate"
fails=()

pg(){ fails+=("$1: $2"); }

# PG-1 — the runtime exists at all. Cheapest possible check, and it has caught a
# capsule whose > Load pointed at a path that was renamed.
[ -f "$RUNTIME" ] || pg PG-1 "runtime missing: $RUNTIME"

# PG-2 — the bridge is LIVE, not a redirect to another command. A bridge that was
# folded into a router and never unfolded still passes PG-1.
if [ -f "$BRIDGE" ]; then
  grep -q "capsules/idea/$NAME/CAPSULE.md" "$BRIDGE" || pg PG-2 "bridge does not load its own capsule"
  grep -qiE 'deprecated|use instead|moved to' "$BRIDGE" && pg PG-2 "bridge reads as a redirect"
else pg PG-2 "bridge missing"; fi

# PG-3 — NEVER OVERWRITE. The guarantee with teeth: it must say so, and there must
# be no escape hatch. `--force` existing anywhere in this runtime is a failure.
if [ -f "$RUNTIME" ]; then
  grep -qiE 'never overwrit|skipped and reported' "$RUNTIME" || pg PG-3 "no never-overwrite guarantee"
  grep -qE 'test -f .*post\.md.*SKIP|SKIP .*already exists' "$RUNTIME" || pg PG-3 "guarantee stated, guard not shown"
  grep -qE '[-]{2}force' "$RUNTIME" && pg PG-3 "a --force escape exists"
fi

# PG-4 — NEVER CONTACTS THE REMOTE. Asserted two ways: the claim, and the absence
# of any call that would contradict it.
if [ -f "$RUNTIME" ]; then
  grep -qiE 'local only|never contacts' "$RUNTIME" || pg PG-4 "no local-only guarantee"
  grep -qE 'POST /v1|api\.medium\.com|--live' "$RUNTIME" && pg PG-4 "runtime references a remote create"
fi

# PG-5 — the per-run cap is a NUMBER in the document, not an adjective.
if [ -f "$RUNTIME" ]; then
  grep -qE 'cap.*\b5\b|default.*\b5\b|--count' "$RUNTIME" || pg PG-5 "no per-run cap"
fi

# PG-6 — the router dispatches HERE rather than reimplementing. A router that grew
# its own copy of this logic is two implementations, and only one gets fixed.
if [ -f "$ROUTER" ]; then
  grep -q "$NAME" "$ROUTER" || pg PG-6 "router does not dispatch to $NAME"
fi

if [ ${#fails[@]} -eq 0 ]; then echo "$NAME: PASS (PG-1..PG-6)"; exit 0; fi
printf '%s: FAIL\n' "$NAME" >&2
printf '  %s\n' "${fails[@]}" >&2
exit 1

Run against the real command:

$ bash capsules/idea/idea-medium-post-generate/verify/verify.sh
idea-medium-post-generate: PASS (PG-1..PG-6)
$ echo $?
0

And run against the red fixture, which is a copy of the runtime with the never-overwrite guarantee removed — one deleted paragraph, exactly the edit a compression pass would make:

$ bash .../verify.sh --runtime tests/.../red/idea-medium-post-generate.md
idea-medium-post-generate: FAIL
  PG-3: no never-overwrite guarantee
  PG-3: guarantee stated, guard not shown
$ echo $?
1

Two findings from one deletion, both PG-3, nothing else firing. That last part is the property to protect: if the red fixture also tripped PG-5, then PG-3 could be deleted from the checker entirely and the fixture would still exit 1 — the exit code would stay honest while the meaning quietly changed.

How we proved it#

The green case passes and reports which assertions ran:

idea-medium-post-generate: PASS (PG-1..PG-6)

Naming them is not cosmetic. PASS alone is the same output a checker with five assertions deleted would produce.

The red fixture fails on PG-3 and only PG-3. One guarantee removed, two findings, both in its own class. That is the check I run when adding a seventh assertion, because a new assertion is the most likely way to make an existing fixture fail incidentally.

PG-4 is asserted in two directions and that was deliberate. Checking that the document claims to be local-only is weak — the claim is prose. Checking that it contains no reference to the remote create path is the half that would catch a runtime which grew a convenience "and also draft it" step. Either alone passes a runtime that contradicts itself.

What I got wrong: PG-2 started as a file-existence check. A bridge that had been folded into a router, left behind as a stub, and never unfolded passed the original PG-2 while the command was unreachable. The assertion now requires the bridge to load its own capsule and to not read as a redirect — and the thing that found that gap was not the fixture, it was the command failing to run.

What I cannot claim is coverage of the contract. Six assertions cover four promises plus two structural properties. The promises are checked by looking for the shape of their enforcement in a document, which is a proxy for the enforcement actually happening when a model interprets it. A runtime that states the guard and whose guard a model skips passes all six.

And a limit worth being plain about: this is a grep over prose. It catches deletion well, because deletion removes the text. It catches a weakened guarantee poorly — a guard rewritten to skip only when the file is non-empty would still match test -f .*post\.md.*SKIP, and PG-3 would pass on a command that overwrites every scaffold.

What this generalizes to#

When an artifact's guarantees are prose and the artifact is edited for unrelated reasons, the guarantees decay silently and the prose keeps asserting them. The fix is an assertion per guarantee, living with the artifact, plus one fixture that removes a guarantee so you know the assertion can fail.

The second half is the part that gets skipped, and skipping it produces the worst available outcome: a checker that passes, a contract that reads as enforced, and no evidence that the two are connected. A check that has never failed is not a check yet — it is a line that has not been contradicted.