The corpus your generator reads is production code

A code generator produces its output by reading a set of guides — worked examples and conventions describing how each layer of the codebase is written. Those guides are prose files in a directory.

They are the most load-bearing files in the system, and they are the easiest ones in it to edit carelessly.

The case this post is about#

One hand-edit, to one guide in the corpus a generator learns from.

The edit was correct. It fixed a real convention the generated code was getting wrong, it was reviewed, and the next generation run produced better output.

It was also untracked by the thing that maintains that corpus, so the next sync from the template rewrote the file and the edit was gone — along with the convention it encoded, silently, on a run that reported success. Nothing anywhere recorded that a surface had been edited by hand or that the edit had been dropped.

This post follows that one edit, because the corpus looks like documentation and behaves like generated code, and the sanctioned way to change it is to change the template and fold.

The symptom#

A generator produces code in an old style. The convention changed months ago, the template it is supposed to follow was updated, and the output still uses the previous shape — because the guide the generator actually reads was never folded forward and has been teaching the old pattern the whole time.

The second thing is a hand-edit that vanishes. Somebody notices the guide is wrong, fixes the paragraph directly, and the generator improves immediately. At the next re-sync from the template, the file is regenerated and the fix is gone — no conflict, no warning, because the edit had no provenance and the sync had no reason to preserve it.

The third is that the guide and the template drift apart silently. Both are correct documents; one describes how the template writes a module and the other is the template. Nothing compares them, so the gap opens gradually and the only symptom is output that is subtly unlike the codebase it is supposed to match.

The fourth is the one that makes this worth a post rather than a convention. Everybody treats these files as documentation, because they are prose, in a docs-shaped directory, written for a reader. They are not documentation. They are the input to a program that writes production code, which makes them source — and nothing about where they sit or what they look like says so.

The consequence of that misfiling is not subtle once you look for it. These files get no review discipline, no provenance, no staleness check and no test, because none of those habits attach to prose — and meanwhile every line of generated code in the repository descends from them.

Why the obvious fix failed#

Keep the guides updated by hand. The thing that produces the drift. It is the right instinct and it depends on somebody noticing that a template change implies a guide change, which is a connection nobody is positioned to see: the person changing the template is not thinking about a corpus in another repository.

Generate the guides from the template automatically. Correct in direction and it destroys the part that matters. A guide is not a dump of the code; it is a worked explanation with chosen examples and stated reasoning. Regenerating it mechanically produces something accurate and useless for its actual purpose.

Review guide changes like code. Right, and it does not fire, because a hand-edit to a prose file in a docs directory does not read as a code change to anybody — including the person making it. The review habit follows the file's apparent category, and its apparent category is wrong.

Point the generator at the template directly. Removes the corpus and the drift with it, and loses the explanation. The generator does better with a worked guide than with raw source, which is the entire reason the corpus exists; removing it to solve a synchronisation problem trades the problem for worse output.

The trade is worth stating plainly because it is the fork in the road. You can have a corpus that is trivially current and teaches poorly, or one that teaches well and can drift. Everything in this post is an attempt to keep the second while removing its failure mode, and the reason the first is not simply chosen is that the quality difference in the output is large and immediate.

The actual mechanism#

Treat the corpus as generated-with-judgement: written only through a sanctioned fold that takes a template change and produces a corpus change, with provenance recorded.

source      the template is where a convention actually changes
fold        one command folds a template change into the corpus, additively
stamp       each surface records what template commit it was folded from
refuse      a direct hand-edit fails, mechanically

The third line is what makes the whole thing checkable. If each guide records the commit it was derived from, then "is this corpus current" becomes a comparison between two identifiers rather than a judgement about prose — and a generator run can refuse to proceed against a stale corpus instead of quietly emitting last quarter's conventions.

The fourth line is the part people resist, and it is the one that holds. A hand-edit is not forbidden because it is wrong — it is usually an improvement — but because it is invisible to the provenance chain, so it survives exactly until the next sync and nobody learns why their fix disappeared.

The sub-case that has to be designed rather than discovered is the escape hatch for the fold itself. The fold writes to the corpus, so a blanket refusal would block the sanctioned path along with the unsanctioned one. What works is a short-lived, explicitly granted consent that the fold holds while it runs: the human confirms, the window opens, the fold writes, the window closes on its own. A permanent environment flag would be the same hole with a lock on it.

The fix#

change      conventions at the template, never at the corpus
fold        with one command, additively, recording provenance per surface
verify      the corpus is current before any generation run
refuse      direct writes, with the fold holding a short consent window

The fold is the only writer and the staleness check is what makes "the corpus is the source" checkable rather than asserted:

/** The corpus must be current with the template HEAD before any generation runs. */
export async function assertCorpusCurrent(templateRef: string, corpus: Surface[]) {
  const stale: string[] = [];
  for (const s of corpus) {
    if (!s.module) continue;                    // cross-cutting: digest-anchored elsewhere
    const anchor = await treeHash(templateRef, s.module);
    if (anchor !== s.anchor) stale.push(`${s.id} (${s.module}: ${s.anchor.slice(0,8)} -> ${anchor.slice(0,8)})`);
  }
  if (stale.length) {
    throw new Error(
      `training corpus is stale against ${templateRef}:\n` +
      stale.map((l) => `  ${l}`).join("\n") +
      `\n  run the retrain fold — generators are currently reading last month's conventions`);
  }
}

/** Direct writes are refused; the fold grants a short, self-expiring window. */
export function assertFoldConsent(latch: string) {
  const age = Date.now() - statSync(latch).mtimeMs;
  if (!existsSync(latch) || age > 30 * 60_000) {
    throw new Error("corpus edits go through the retrain fold — a hand edit is provenance-untracked");
  }
}

A generation run against a template that moved:

$ generate profile
training corpus is stale against upstream/dev:
  CORE_STORE (core/store: 0c57da24 -> 9f2c1b05)
  CORE_DATA  (core/data:  7f2a9c41 -> 243a9b3d)
  run the retrain fold — generators are currently reading last month's conventions

The failure it prevents is subtle: nothing errors without this check. The generator produces perfectly good code against conventions that were replaced, and the drift only surfaces when somebody notices the output no longer matches the template it was supposed to follow.

The third line is the one that converts this from hygiene into a gate. A generator that checks its corpus against the template before running will either proceed or tell you exactly what to re-fold, which means a stale corpus produces a stopped build rather than a quietly wrong file.

What makes the discipline survivable is that the fold is additive. It does not rewrite the guide from scratch, so the worked examples and the reasoning are preserved; it appends what the template change implies. That is the property that lets the corpus stay a document a person wrote while also being derived.

How we proved it#

The drift is a comparison and therefore exact. Each surface records a template commit; the template has a current commit; the two either match or they do not. There is no heuristic in that, and it turns a question that used to be answered by reading prose into one answered by comparing two strings.

The hand-edit failure mode is reproducible deliberately. Edit a guide directly, run a re-sync, and watch the edit disappear with no conflict and no message. That takes a minute, and it is a far better argument for the discipline than any amount of explaining, because the disappearance is silent and surprising in exactly the way the real incident was.

What I cannot tell you is how much bad output the drift caused. I know the corpus was stale and I know what it was teaching, and I have no way to attribute specific generated files to specific stale paragraphs after the fact. The honest claim is that the mechanism was clearly wrong, not that I measured its cost.

What I have not solved: the corpus that is wrong on its own terms. Everything above keeps the guides synchronised with the template; none of it notices that a worked example teaches a pattern that was never good. A guide can be perfectly current and quietly wrong, and the only check for that is somebody reading it with judgement — which is the thing the provenance machinery cannot substitute for.

The two failure modes want different remedies and are easy to conflate. Staleness is a synchronisation problem and yields to automation completely. A bad example is an editorial problem and yields only to somebody reading the corpus as prose, with the question "would I want code that looks like this" — which is a review of the guides themselves, on its own cadence, and nothing in the fold machinery prompts it.

And a cost worth stating: the fold is slower than editing the file. Somebody who can see the wrong sentence and the right one has to go and change the template instead, confirm, and run a command, and that is minutes rather than seconds. The discipline is paid every time and the benefit arrives only at the next sync, which is a terrible ratio in the moment and the right one over a year.

The way I have made it bearable is to shorten the loop rather than argue with the cost. When the fold is one command and takes seconds, the minutes are mostly the walk to the template; when it is a multi-step process somebody has to recall, the discipline loses to the hand-edit every time and deserves to.

The thing that changed how I think about it is the realisation that "is this a document or is this code" is decided by what reads it, not by what it looks like. These files are prose, written for humans, and a program consumes them to produce production source — which makes every property I care about in code apply to them: provenance, review, drift detection, and a path by which changes arrive.

The reverse test is useful too, and it is how I now spot these: look for files that are edited casually and consumed rigorously. A mismatch in either direction is worth a second look, but that one — careless writes, precise reads — is where the expensive surprises live.

What this generalizes to#

An artifact's category is set by its consumer, and a file that looks like documentation while being read by a program is the most reliably mishandled thing in a codebase.

The pattern shows up wherever prose becomes input. A prompt file that drives behaviour. A README whose code blocks are extracted by a build step. A changelog parsed to compute a version. A policy document a rules engine reads. A spreadsheet somebody exports into a deployment. In each case the handling the file receives matches its appearance rather than its role, and the gap is where untracked changes live.

So the useful question about any file is what reads this, and does the way it is handled match that — because the moment a program depends on a document, the document has become source, and everything you would do for source applies whether or not it looks the part.