Is this derived thing current? Make it a comparison

A file is generated from a specification. Months later somebody needs to know whether it still reflects that specification, and the available evidence is a modification time, a convention, and whoever happens to remember.

All three are worse than they look, and the modification time is the worst of them.

The case this post is about#

One generated artifact and the question asked of it every run: is this still current?

inputs      the declared set of source files this artifact derives from
digest      a hash over that set's contents, computed one documented way
sidecar     the digest, stamped beside the output at generation time
check       recompute and compare — equal is current, different is stale

The version this replaced compared modification times. It reported the artifact stale after a git checkout that changed nothing, reported it current after an edit that was reverted and re-applied, and could not answer the question at all on a machine that had cloned the repo fresh.

This post follows that one artifact, because timestamps answer when was this touched and the question is what was this made from — and the second has a stable answer that survives a clone.

The symptom#

The honest version of the question is "was this regenerated after the last change to its inputs", and the artifact has no opinion about it. It contains output. It does not contain any statement about what it was made from, so the question has to be answered from outside — by someone who was there, or by a signal that only approximates it.

The second thing is that the approximation is usually a timestamp, and timestamps describe the file system rather than the content. A fresh checkout rewrites them all. A clone sets them to now. A formatter that changes nothing touches them. A file copied from one machine to another arrives with whatever the copy decided. Every one of those events moves the signal without moving the thing it is standing in for.

The third is that the convention does hold for a while. "Regenerate after editing the spec" works while the people who wrote it are the ones doing the editing. It decays in the ordinary way: a change made in a hurry, a contributor who did not know, a rebase that reintroduced an old version of the source and not the output.

The fourth is how it surfaces. Not as an error — a stale generated file is perfectly valid, compiles, passes its tests and is internally consistent. It is simply describing an earlier version of something, and the only way to find out is to regenerate and look at the diff, which is exactly the work the question was trying to avoid.

That last property is what makes it expensive. The failure has no symptom of its own; it shows up later as a contradiction between two parts of the system that were each individually correct, and tracing that contradiction back to a file nobody regenerated takes far longer than the regeneration would have.

Why the obvious fix failed#

Compare modification times. The classic, and it is wrong in both directions. It reports stale after a checkout that changed nothing, which trains people to ignore it, and it reports current after any operation that happened to touch the output last — which is the direction that costs you.

Regenerate every time. Removes the question by doing the work unconditionally. Fine when generation is cheap and deterministic; untenable when it costs minutes, calls a model, or produces output that differs harmlessly between runs, because then every build has a diff and nobody can tell a real change from noise.

Stamp a version number. A field in the output saying which version of the spec it came from. Better, and it depends on somebody bumping the version when they edit — which is the same memory the convention relied on, relocated into a field that now looks authoritative.

Let CI regenerate and diff. Genuinely good, and it has a prerequisite that is easy to miss: the generator has to be runnable in CI, deterministic, and fast enough to run on every change. When any of those fails the check gets disabled, and a disabled check is worse than none because its absence is not visible.

The actual mechanism#

Record, inside the output, a digest of the inputs it was made from. Currency then becomes a comparison: hash the inputs now, compare against what is recorded, and the answer is a fact rather than an inference.

inputs      the declared set of source files this artifact derives from
digest      a hash over that set's contents, computed one documented way
sidecar     the digest, stamped beside the output at generation time
check       recompute and compare — equal is current, different is stale

The property that makes this better than every alternative above is that it is content-addressed rather than time-addressed. A checkout does not change content. A formatter that reorders nothing does not change content. Copying a file does not change content. The signal moves when and only when the thing it describes moves, which is the whole of what a staleness check is supposed to do.

It also moves the human out of the loop entirely. Nobody has to remember to bump anything, because the digest is computed from the files rather than declared about them, and a change that somebody forgot to mention changes it anyway.

The sub-case that determines whether this works at all is which files count as inputs. The digest is only as good as the set it covers, and that set has to be declared explicitly — a list, or a set of globs — because an input nobody listed is a change the check cannot see. This is the same enumeration problem that every coverage check has, in a new place.

And there is a specific trap inside it that is worth more than a sentence: a glob matching nothing hashes perfectly stably. A pattern pointing at a directory that has since been renamed produces an empty set, an empty set produces the same digest forever, and the check reports current on every run with total confidence. It is the most dangerous possible failure for a verification mechanism, because it is indistinguishable from success and it gets quieter over time rather than louder.

The remedy is small and has to be deliberate: refuse a digest computed over zero files. Not warn — refuse, because a warning in a passing run is read by nobody. A glob that matches nothing is either a typo or a stale declaration, and both are the sort of thing you want to hear about on the day they happen rather than six months later when somebody asks why the gate has never once fired.

The fix#

declare   the input set, as explicit paths or globs, beside the generator
hash      that set's contents, with a documented ordering and normalisation
stamp     the digest into a sidecar written at generation time
gate      recompute, compare, and fail on mismatch — or on an empty match

Both halves, and the zero-match refusal is not an afterthought — it is what keeps the other one honest:

type Decl = { id: string; globs: string[] };

async function filesFor(d: Decl): Promise<string[]> {
  const out: string[] = [];
  for (const g of d.globs) for await (const e of expandGlob(g)) if (e.isFile) out.push(e.path);
  // A glob matching NOTHING hashes stably forever — the most dangerous possible "pass".
  // Refuse, never warn: a warning on a green run is read by nobody.
  if (out.length === 0) throw new Error(`${d.id}: globs matched 0 files — typo or stale declaration`);
  return out.sort();                   // fixed order, or the digest moves between machines
}

const normalise = (s: string) => s.replace(/\r\n/g, "\n").replace(/[ \t]+$/gm, "");

export async function digestOf(d: Decl): Promise<string> {
  const parts = await Promise.all(
    (await filesFor(d)).map(async (p) => `${p}\n${normalise(await Deno.readTextFile(p))}`),
  );
  const h = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(parts.join("\0")));
  return [...new Uint8Array(h)].map((b) => b.toString(16).padStart(2, "0")).join("");
}

export async function checkCurrent(d: Decl, sidecar: string) {
  const want = JSON.parse(await Deno.readTextFile(sidecar)).digest;
  const got = await digestOf(d);
  if (got !== want) {
    throw new Error(`${d.id} is STALE — regenerate (${want.slice(0, 8)} → ${got.slice(0, 8)})`);
  }
}

Three runs, including the one a timestamp gets wrong in both directions:

$ touch guides/CORE_STORE.md && check CORE_STORE
ok                                  # content unchanged — a timestamp would have said stale

$ echo " " >> guides/CORE_STORE.md && check CORE_STORE
error: CORE_STORE is STALE — regenerate (0c57da24 → 9f2c1b05)

$ check GAME_SCENE                  # its glob points at a directory that was renamed
error: GAME_SCENE: globs matched 0 files — typo or stale declaration

The second line hides the only real design decision, which is normalisation. Hash raw bytes and you get false positives from line endings and trailing whitespace; normalise aggressively and you risk hashing away a change that mattered. The choice has to be made once, written down, and used identically by the generator and the gate — because two implementations that disagree produce a permanent mismatch that looks exactly like permanent staleness.

Ordering is the same kind of trap in miniature. A digest over a set of files has to fix the order it reads them in, or it changes when the file system changes its mind about directory listing, and you get a check that fails on a different machine for no reason anybody can see.

The fourth line is where the value lands, and the "or" in it is doing real work. A gate that fails on mismatch is the obvious half; a gate that also fails when the input set is empty is the half that keeps the first one honest. Without it, the way this mechanism dies is not dramatic — the declaration drifts, the match goes to zero, and the gate spends the rest of its life passing.

How we proved it#

The timestamp failure is a two-command demonstration. Touch a source file without editing it and ask the timestamp check what it thinks: it reports stale, and nothing changed. Then edit the source and touch the output: it reports current, and the output is wrong. Both directions, in about a minute, and after that nobody argues for mtime.

The digest version passes both of those. A touch moves nothing because content is unchanged; an edit of one character moves the digest and the gate fires. That is the property, stated as a test rather than as a claim, and it is cheap enough to keep as a regression check.

The empty-glob trap we found by building the failing case on purpose. Point a declaration at a directory, rename the directory, and run the gate: it passes. Seeing a verification mechanism report success over nothing at all is more persuasive than any argument for the zero-file refusal, and it is the test I would write first on any check of this shape.

What I have not solved is that a digest tells you something changed, not whether it mattered. A comment edited in a source file invalidates every artifact derived from it, and the regeneration that follows produces an identical output. The check is correct and the work is wasted, and distinguishing semantic change from incidental change would mean parsing the inputs — which is a much larger mechanism than this one and I have not built it.

And a limitation worth naming: this verifies that the output was generated from these inputs, not that the generator was behaving correctly at the time. A digest match means the artifact is current with respect to its sources; it says nothing about whether the generation was right. Those are different questions, and this mechanism answers the smaller one cleanly rather than the larger one vaguely — which I think is the correct trade, provided nobody reads the green as more than it is.

The thing that changed how I think about it is that currency is a relation between two things and we habitually store it as a property of one. "Is this current" is really "does this agree with that", and every mechanism that tries to answer it from a single file — a timestamp, a version field, a flag — is compressing a comparison into an attribute, and losing exactly the information the comparison was carrying.

Once that is framed as a comparison, the implementation almost writes itself, and the remaining work is all in the input declaration rather than in the hashing. Which is the usual shape: the clever part is trivial and the boring part — saying precisely what the thing depends on — is where the failures live.

What this generalizes to#

Anything derived has a relationship to what it was derived from, and most systems do not record it.

The pattern repeats wherever an artifact outlives its inputs. A lockfile and its manifest. A compiled schema and the schema. Documentation generated from code. A cached computation. A trained corpus and the source it was folded from. In each case the derived thing is valid in isolation, so nothing goes red, and the only question that matters — is this still true of its source — has no answer stored anywhere.

So the test is to ask, of any generated file: if its input changed yesterday, what in the system would know? If the answer is a person, a timestamp, or a convention, then the honest state is that nobody knows — and the fix is not more discipline but one recorded digest, which costs a line at generation time and converts a memory into a comparison.