Editing the generated thing is how the spec stops meaning anything

A request arrives: add a field, add a screen, change what a button does. The spec describes all of this, and the source is generated from the spec.

The change is made in the source, because that is where the change is.

The case this post is about#

One feature, built straight into the source because it was small:

intent    arrives as a request, in prose, from a person
spec      the first place it is materialised — screens, flows, states, strings
source    generated from the spec; never the place a change begins
reports   coverage, tests, translations, audits — all computed from the spec

The feature worked. It also did not exist as far as the fourth line was concerned: the coverage report did not count it, the translation audit did not know its strings, the navigation check did not see its route, and the next regeneration had no reason to keep it.

This post follows that one feature, because the shortcut is genuinely faster and the thing it skips is most of what the project knows about itself.

The symptom#

The feature works. It is in the running app, it is correct, somebody uses it. And nothing else in the project knows it exists.

The second thing is the list of downstream things that were computed without it. The coverage report counts screens from the spec, so the new one is not in the denominator. The test matrix enumerates flows from the spec, so nothing tests it. The translation export walks the spec's strings. The navigation audit checks the spec's routes. Every one of those is still green, and every one of them is now green about a smaller project than the one that exists.

The third is that this is invisible from both sides. From the source, the feature is ordinary code with nothing marking it as unaccounted-for. From the spec, there is simply no entry — and an absent entry produces no anomaly, because specs are full of things that do not exist yet.

The fourth is the regeneration, which arrives eventually and is the loudest possible version of the problem. A generator run that rewrites a file removes a feature nobody recorded, and because the feature was never in the spec there is no way to tell whether its disappearance was intended. It looks exactly like a file being brought back into line.

The lesson people take from that moment is usually the wrong one: that regeneration is dangerous and should be avoided. That conclusion is locally reasonable and it ends with the generator switched off, the spec abandoned, and a codebase maintained by hand with a large amount of unused machinery attached to it.

Why the obvious fix failed#

Update the spec afterwards. The honest intention, and it is a second task with no forcing function. The feature already works; the spec update produces nothing visible; it goes to the bottom of a list and competes with work that does.

Generate the spec from the source. Appealing because it makes the shortcut harmless, and it inverts the whole arrangement. A spec derived from the source cannot express intent that has not been built yet, which is most of what a spec is for — it becomes a description rather than a plan.

Review for spec-first. Asks the reviewer to notice an absence in a different file from the one they are reading. Reviewers are reading a diff; the missing spec entry is not in it, and nothing in the diff suggests it should be.

Block source edits entirely. Correct in direction and impossible as stated, because plenty of source is not generated and the boundary between the two is not obvious to somebody mid-task.

Keep a checklist in the pull request template. The cheapest thing anybody can try, and it works for about two months. A checklist item that can be ticked without doing the thing converges on being ticked without doing the thing, and it leaves behind a record implying the opposite.

The actual mechanism#

Make the spec the entry point for intent, and make everything downstream derive from it — so the shortcut does not merely violate a convention, it skips the step that makes the work visible.

intent    arrives as a request, in prose, from a person
spec      the first place it is materialised — screens, flows, states, strings
source    generated from the spec; never the place a change begins
reports   coverage, tests, translations, audits — all computed from the spec

The usual framing is that the spec is "the source of truth", which is true and does not explain why anybody should care. The operational reason is the fourth line: the spec is what every other mechanism reads. A feature that is not in it is not merely undocumented — it is outside the reach of every check, count, and export the project has.

That reframing also makes the cost concrete. Skipping the spec does not save the work of writing it; it defers the work and silently cancels the benefits in the meantime, which are the benefits most people assume they still have because the reports are green.

The sub-case worth stating is the change that seems too small to materialise. A copy fix, a default value, a tweak to an existing handler. These feel absurd to route through a spec, and they are where the practice dies, because the exception for small things has no edge — every change is small from inside, and the second one is always justified by the first.

There is a shape of this that is harder to see and more common than the straightforwardly skipped feature: the change that is materialised partly. The screen is added to the spec and the string is not; the flow is updated and the state is not. These produce reports that are not wrong in an obvious way, just slightly short, and slightly short is the hardest kind of wrong to notice.

Partial materialisation also defeats the obvious detection. A gate that asks whether the spec was touched is satisfied by the screen entry, and the missing string is as invisible as it would have been with no spec change at all — so the check that catches the blatant case passes the subtle one, which is the wrong way round for a mechanism that is supposed to be a backstop.

The fix#

materialise   every change starts as a spec change, including small ones
generate      the source follows; the generator is the only writer
derive        every report, export and audit reads the spec, not the source
gate          a source change with no corresponding spec change fails

The gate is a diff question, and its message is the half that decides whether people route through the pipeline or around the gate:

const GENERATED = ["core/dto/", "core/store/", "feature/"];
const specFor = (p: string) => `design/${featureOf(p)}/spec.yaml`;

export function assertSpecFirst(changed: string[]) {
  const src  = changed.filter((p) => GENERATED.some((g) => p.startsWith(g)));
  const spec = changed.filter((p) => p.startsWith("design/"));
  if (src.length === 0) return;

  const unbacked = src.filter((p) => !spec.includes(specFor(p)));
  if (unbacked.length === 0) return;

  throw new Error(
    `source changed with no spec change behind it:\n` +
    unbacked.map((p) => `  ${p}\n    materialise in ${specFor(p)}, then regenerate`).join("\n") +
    `\n\n  the spec is what every report, export and audit reads. a change that\n` +
    `  skips it works, and is invisible to all of them.`,
  );
}

On a change that edits generated source directly:

$ ./gradlew verifySpecFirst
source changed with no spec change behind it:
  core/store/ProfileStore.kt
    materialise in design/profile/spec.yaml, then regenerate

  the spec is what every report, export and audit reads. a change that
  skips it works, and is invisible to all of them.

The partial case is the one this misses, and it is worth knowing: a change that touches the spec for the screen but not for the string passes, because spec.includes(...) is satisfied. Catching that needs a field-level comparison, which I have not built.

The fourth line is the only part that holds under pressure, and it has to be mechanical for the usual reason: the shortcut produces working software, so nothing else will ever object to it. A convention here loses to every deadline, because taking it is never the thing that breaks.

The gate's message matters as much as the gate. "Source changed without a spec change" is a fact; "this change belongs in this spec file, run this to regenerate" is an instruction, and the difference determines whether people route through the pipeline or route around the gate.

What makes the whole arrangement affordable is that materialising has to be cheap. If adding a field to the spec is a twenty-minute ceremony, the rule is a tax and will be avoided; if it is a few lines and a regeneration, it is simply how the work is done. The ergonomics are not a nicety here — they are the enforcement mechanism that actually works, with the gate as a backstop.

The ordering between those two is not optional either. A gate introduced before materialising is cheap produces a team that resents the pipeline and a steady trickle of justified exceptions; the same gate introduced afterwards is barely noticed, because it only fires on work that was already taking the awkward path.

How we proved it#

The invisibility is directly demonstrable. Add a screen in source only, then run the coverage report: unchanged, still complete, still green. That takes ten minutes and it is the single most convincing thing I can show anybody who thinks the spec is paperwork.

The regeneration test makes the fragility concrete. Make a source-only change, regenerate the file, and watch it disappear with no warning and exit zero. People who have seen that once stop arguing about spec-first.

What I have not measured is how much slower spec-first actually is per change. It is clearly slower for a one-line fix and clearly faster by the second regeneration; I have not instrumented where the crossover sits, and the honest answer is that I adopted it on the strength of the failures rather than a measurement.

The partial case is the one I would instrument first if I were starting again. The fully skipped spec gets caught by any gate at all; the half-materialised change passes the gate, dilutes the reports slightly, and accumulates — and I found those by noticing a count that felt low rather than by any mechanism, which is not a method.

What I have not solved: the spec's own expressiveness. Some things are genuinely awkward to say in a structured format — an unusual interaction, a platform-specific workaround — and the spec-first rule makes those changes disproportionately expensive. The answer is supposed to be extending the spec's vocabulary, and that is a real cost I keep paying.

And a limitation worth naming: the spec being the entry point does not make it correct. A project can materialise everything faithfully into a spec that describes the wrong product, and every report will agree with it perfectly. This arrangement buys coherence between the spec and the system; it says nothing about whether the spec is any good, and the green reports are about agreement rather than about quality.

The thing that reframed it was noticing that the spec's value is proportional to how much reads from it. A spec nothing derives from is documentation, and documentation drifts because nothing depends on it. A spec that generates the source, the tests, the translations and the reports cannot drift silently, because everything downstream of it moves when it moves.

Which means the way to make spec-first stick is not to insist on it, but to move more things downstream of the spec. Every report that starts reading from it raises the cost of skipping it, and at some point the shortcut stops being attractive without anybody having to be told.

That also gives a way to sequence the adoption, which otherwise looks like an all-at-once change. Move one report at a time — the one people check most often first — and each move makes the next easier to justify, because the spec is visibly load-bearing by then rather than aspirationally so.

What this generalizes to#

A system computes its reports from a model, and anything not in the model is outside every answer it gives.

The same shape turns up wherever a model is meant to describe a system: infrastructure declared in code and a resource created in the console, a dependency graph and a library vendored by hand, a schema and a column added directly. In each case the real thing is bigger than the model, and the model's outputs are confidently, quietly, about the smaller version.

So the question worth asking of any such model is what is in the system that is not in here, and what would tell me. If nothing would, then the model's completeness is an assumption rather than a property — and every number computed from it inherits that assumption without declaring it.