AI-Driven Development

The prompt is not the artifact — the runtime is

A behaviour defect belongs in the command's runtime file, not in better instructions next time. What to edit, how to lock it, and where that stops working.

MobileByteSensei 9 min read
A conversation that ends, beside a file that is loaded again on every run
aillmpromptclaude-code

An agent does something wrong. You explain why it was wrong, it agrees, it does the right thing.

Nothing has been fixed. The next run starts from the same file you did not change.

The case this post is about

One recurring problem, fixed three times:

The first fix was a better prompt — I explained the constraint again, in the conversation, and the next output was correct. The second was a note in a documentation file recording the measurement and the date, which is a real artifact and which nothing in any run opens. The third moved the same sentence into the contract file that every run loads.

Only the third one changed what happens next time. The first two were correct, well-reasoned, and invisible to the thing they were meant to govern.

This post follows those three fixes, because they are increasingly durable and only one of them is enforced — and the difference between the second and the third is not the content of the sentence but which file it is in.

The symptom

The fix that feels like a fix is the conversational one. You correct the behaviour, the correction lands, the work continues — and the correction lives in a transcript that the next invocation has never read.

This is worth being precise about, because it is not a memory problem. A fresh run is not forgetting your correction; it never had it. The only things it has are the files it loads. So the question "how do I stop this happening again" has exactly one honest answer: which file would have prevented it, and what does that file say now.

The symptom is a class of defect that recurs with a slightly different surface each time — a step skipped, a scope narrowed, an irreversible call reached — and a transcript history full of corrections, each of which worked once.

Why the obvious fix failed

The obvious fix is a better prompt: more explicit instructions at the start of the next session.

It fails on durability. A well-crafted instruction that lives in your head, or in a message you sent on Tuesday, protects exactly the sessions in which you remember to send it. The failure rate is the rate at which you forget, which is not low, and it is highest when you are busy — which is when the defect is most expensive.

The second attempt is an apology-shaped commitment. I'll be careful about that. I have written this, and it is the least useful sentence available, because it names no mechanism. Nothing in the system is different afterwards.

The third — and this one is genuinely tempting — is a one-off workaround at the call site. Pass an extra flag. Add a guard clause in the one place it bit you. It works and it treats the symptom: the same defect reappears in the next caller, which has a different path to the same root.

The actual mechanism

Fix it in the artifact that is loaded on every run, at the root, and then lock it with something that fails.

For a command-shaped agent, that is a small, specific set of places:

Where a behaviour fix can live, and how long each one lasts

The rule I work under states it as a sequence, and the ordering is the part that matters: edit the runtime at the root cause, then lock it with a rule, an evaluator, and a red-and-green fixture — so the defect class is machine-checkable and cannot regress. An apology, a promise, or a one-off workaround are all explicitly not responses.

The four steps of a durable heal, and the two responses that are not one

There is a clause for the case where it happens again, which I find the most useful part: if the same class is flagged a second time, the first heal was symptom-level, and the response is to escalate to a stronger guard rather than to apologise again. A repeat is diagnostic information about the fix, not about the agent.

The fix

What this looked like in practice, across one session, three times.

A contract file. Composited covers kept colliding with the artwork. The durable fix was not to remember shorter headlines — it was a line in the brief contract stating that each headline line has roughly a 26-character budget before it crosses the point where the background scrim clears, with the measurement and the date attached.

A generator. A guard was refusing legitimate work because it matched a protected path anywhere in a command, including inside replacement text. The fix went into the hook itself, behind nine red-first cases, rather than into a habit of phrasing commands differently.

A build script. A scaffolded post was serving a placeholder body on the live site. The fix was a status: draft check in both copy branches of the generator — plus a gate that counts the guards, because fixing one branch is what a hurried change looks like.

None of those is a better prompt. All of them change what the next run loads.

And the thing that makes the distinction operational is that it is checkable. A fix is durable if it lives in a file the next run reads, and that is a property you can assert rather than intend:

type Fix = {
  symptom: string;
  landedIn: string | null;      // null = it only ever existed in a conversation
  guardedBy: string | null;     // the check that makes it stay
};

type Tier = "always-loaded" | "conditional" | "reference" | "not-loaded";

function tierOf(path: string): Tier {
  if (/^core\/rules\/always\//.test(path)) return "always-loaded";
  if (/^(layers|capsules)\/.*\.md$/.test(path)) return "conditional";
  if (/^docs\//.test(path)) return "reference";
  return "not-loaded";
}

function classify(fix: Fix, triggerFires: boolean) {
  if (fix.landedIn === null) {
    return { durable: false, why: "re-prompt only — the next run loads nothing new" };
  }
  const tier = tierOf(fix.landedIn);
  if (tier === "reference" || tier === "not-loaded") {
    // Real, correct, and never read. The failure mode "put it in the runtime" hides.
    return { durable: false, why: `landed in a ${tier} file — correct and never loaded` };
  }
  if (tier === "conditional" && !triggerFires) {
    return { durable: false, why: "conditional file whose trigger does not match this work" };
  }
  if (fix.guardedBy === null) {
    return { durable: true, why: `loaded (${tier}) but unguarded — a later edit can undo it silently` };
  }
  return { durable: true, why: `loaded (${tier}) and guarded by ${fix.guardedBy}` };
}

The session's four attempts at the same class of problem:

[
  [{ symptom: "cover headline collides", landedIn: null, guardedBy: null }, true],
  [{ symptom: "cover headline collides", landedIn: "docs/guides/BRIEF_NOTES.md", guardedBy: null }, true],
  [{ symptom: "cover headline collides", landedIn: "layers/idea/commands/_shared/brief-contract.md", guardedBy: null }, true],
  [{ symptom: "guard false positive", landedIn: "core/rules/always/RULE-SPEC-001.md", guardedBy: "nine red-first cases" }, true],
].forEach(([f, t]) => console.log(classify(f as Fix, t as boolean)));
{ durable: false, why: 're-prompt only — the next run loads nothing new' }
{ durable: false, why: 'landed in a reference file — correct and never loaded' }
{ durable: true,  why: 'loaded (conditional) but unguarded — a later edit can undo it silently' }
{ durable: true,  why: 'loaded (always-loaded) and guarded by nine red-first cases' }

The second row is the one I would have scored as done. The 26-character budget was written down, with its measurement and its date, in a file that reads like the right home for it — and nothing in any run opens that file, so the next cover collided exactly as before. Moving the same sentence into the brief contract changed nothing about its content and everything about whether it has an effect.

Not every file is loaded every run

"Put it in the runtime" hides a second decision, and getting it wrong produces a fix that is real, correct, and never read.

The command structure I work in has three tiers. A small bridge file is always present. It pulls a body that loads when the command runs. That body, in turn, points at deep reference documents which load only when a particular phase needs them. All three are files on disk, and they have completely different odds of being in context at the moment the defect would occur.

So a fix has to land where the failure happens. A rule about how to write an asset brief belongs in the brief contract, which loads whenever assets are generated. The same rule placed in a general reference document is correct and inert — the phase that needed it never opened the file.

The inverse mistake is worse and more tempting: putting everything in the always-loaded tier, where it is guaranteed to be read. That tier is a fixed budget competing with the actual task, which is why it carries a hard line limit here. Every sentence added to it makes every other sentence in it slightly less likely to be noticed.

The heuristic I use: the fix goes in the narrowest file that is certain to be loaded when the defect could happen. Narrower than that and it will not be read; wider and it dilutes everything that shares the space.

How we proved it

The evidence I trust most is the one that embarrassed me.

I learned, by hitting it twice, that a long emphatic block of forbidden instructions makes the image producer return nothing at all. I wrote the finding into the contract file:

Keep the negative list SHORT — a long emphatic FORBIDDEN block makes the
producer return no screens at all (measured 2026-09-30, twice, same subject).

Then, about an hour later, I wrote a brief with a long emphatic block of geometry requirements in capitals. It returned nothing. Twice.

Four ways of recording a fix, and how many runs each one survives

My own experience of the lesson did not prevent the lesson from recurring, ninety minutes later, in the same session. What eventually stopped it was reading the line I had written in the file.

That is the entire argument, demonstrated against myself: the durable part of a fix is the file, not the understanding. Understanding decays inside a single session; a file does not.

The honest limitation is real, though. A runtime file is prose, and prose in a loaded file competes for attention with the task exactly the way any instruction does. A note I wrote and then violated is evidence for writing it down and also evidence that writing it down is not sufficient. The line that actually stopped the placeholder reaching production was not a sentence in a document — it was a check that returned a non-zero exit code. The document is where the reason lives; the gate is what enforces it.

What this generalizes to

When an agent does something wrong, resist the fix that ends when the conversation does. Ask which file would have prevented it, edit that file at the root rather than at the call site that happened to bite you, and add a check that fails when the fix is absent.

Then treat a repeat as a verdict on your fix rather than on the agent. If the same shape comes back, the heal was at the symptom — and the next version needs to be something that returns non-zero, not something more strongly worded.

One post a week, no digest

New writing when it lands. No roundups, no course pitch, no "10 tips".

Follow by RSS