Every generated file should say what made it

A file is produced by a process with several possible paths through it. The file is correct. A week later, nobody can say which path made it.

The information existed for the duration of one function call and was never written down.

The case this post is about#

One generated image and one question asked a week later: which path produced this?

There were two paths that could have. One calls a design service; the other is a procedural fallback. The artifact is a PNG, and a PNG records nothing about who made it.

So the question got answered by argument — it looks like the service output, the service was the default, therefore it came from the service. All three of those were true and the conclusion was wrong: 303 of 304 assets had come from the fallback.

This post follows that one image, because the sidecar that answers it is a few lines written at creation time, and the thing that makes it work is that it is written by the producing path rather than by whoever is asking.

The symptom#

An argument that cannot be settled. Somebody believes the artifacts were produced one way, somebody else believes another, and the artifacts themselves are mute. The discussion runs on recollection and plausibility, which is a bad way to establish a fact that was perfectly knowable at the moment it happened.

The second thing is that the artifacts look identical regardless. A file produced by the preferred path and one produced by a fallback have the same extension, the same shape, and often very similar content — the whole point of a fallback is to produce something acceptable, so acceptability is not a distinguishing feature.

The third is that logs do not survive long enough. The information is usually in a log somewhere at the time, and logs are rotated, sampled and discarded on a schedule set by storage cost. The artifact is kept because it is the point; the record of how it was made is kept until Tuesday.

The fourth is the question that actually gets asked, which is never "how was this made" in the abstract. It is "are all of these made the right way", asked about a few hundred files at once, usually because something has gone wrong — and that question needs a field to count, not a story to reconstruct.

The difference between those two is the difference between a minute and an afternoon, and it is decided entirely by a choice made before anybody knew there would be a question. That is the awkward economics of provenance: the cost is paid by people who have no problem, and the benefit goes to people who do.

Why the obvious fix failed#

Check the logs. Works for the last few days and not for the ones that matter. The question arrives late by construction: you ask how things were made after noticing something odd, and noticing takes time.

Re-derive it from the content. Sometimes possible and always fragile. Inferring the producer from characteristics of the output is a heuristic that works until two paths produce similar results, which is precisely the case you are trying to distinguish.

Keep a central index. A database of what was produced and how. It works and it is a second source of truth that drifts: an artifact deleted or regenerated outside the tool leaves the index describing something that no longer exists.

Indexes also invite a second failure that sidecars cannot have: partial writes. A tool that produces an artifact and then records it centrally has a window between the two, and a crash in that window leaves an artifact nothing knows about. Writing the sidecar in the same operation as the artifact closes it.

Put it in the filename. Tempting and it does not scale past one field. The moment you want the path, the version, the timestamp and the input hash, the filename becomes a format with a parser, and renaming a file silently destroys the metadata.

The actual mechanism#

Write a small file beside the artifact that records how it came to exist.

sidecar    one file per artifact, same name plus a suffix, written at creation
contains   which path produced it, with what inputs, at what version
travels    with the artifact — copied, moved and committed alongside it
readable   plain text, so the question is answered with a grep

The third line is what makes a sidecar better than an index. Metadata that lives next to the thing it describes moves when the thing moves, gets deleted when the thing is deleted, and arrives in a code review in the same diff. An index has to be maintained in sync with reality; a sidecar is carried by reality.

The fourth line is a deliberate choice against anything cleverer. A format that needs tooling to read gets read by tooling that someone has to write, and the entire value here is that an ordinary search answers the question across hundreds of files in one command.

The sub-case that determines whether a sidecar is trustworthy is where it is written from. It has to be stamped by the branch that actually produced the artifact, not by a caller that believes it knows which branch ran. The further the stamping sits from the decision, the more chances there are for the two to diverge — and a confidently wrong sidecar is worse than none, because it converts an open question into a false answer.

Which is the one way this can be worse than doing nothing. An absent record leaves you uncertain and knowing it; a wrong record leaves you confident and mistaken, and the confidence is supplied by the very mechanism that was meant to make things rigorous.

The fix#

stamp      from inside the producing path, never from the caller's assumption
record     path, inputs, version, timestamp — small, flat, obvious
colocate   beside the artifact, same basename, committed together
grep       the field across the corpus when the question comes up

Stamped from inside the producing function, which is the clause that matters — a sidecar written by the caller records what the caller believes happened:

type Provenance = {
  artifact: string; engine: string; engine_version: string;
  inputs: Record<string, string>; at: string;
};

async function writeSidecar(artifact: string, p: Omit<Provenance, "artifact" | "at">) {
  await Deno.writeTextFile(`${artifact}.gen.yaml`,
    toYaml({ artifact, ...p, at: new Date().toISOString() } satisfies Provenance));
}

export async function renderArt(spec: ArtSpec, out: string) {
  const bytes = await engine.render(spec);          // the producing path
  await Deno.writeFile(out, bytes);
  // Same function, same scope. The record cannot describe a render that did not
  // happen, because it is written where the bytes are.
  await writeSidecar(out, {
    engine: engine.name,
    engine_version: engine.version,
    inputs: { intent: spec.intentPath, palette: spec.palette.join(",") },
  });
  return out;
}

The payoff is a flat grep months later, with no index to keep current:

$ grep -l 'engine: stitch' posts/*/assets/*.gen.yaml | wc -l
     187

$ grep -h 'engine_version:' posts/*/assets/*.gen.yaml | sort | uniq -c
     154 engine_version: 2.1.0
      33 engine_version: 1.9.3        <- predate the palette change; regenerate these

That second query is the one that justifies the whole convention. "Which assets predate the change" is unanswerable without a record, and answerable in one line with it.

The fourth line is the payoff and it is worth being concrete about how small it is. "How many of these were produced by the preferred path" becomes one command over a directory, returns a number, and takes a second. Without the sidecar, the same question is an afternoon of inference that produces an estimate.

What makes this worth doing on everything rather than on the artifacts you expect to care about is that you cannot predict which ones those are. The sidecar is cheap at creation and impossible to backfill — the information is gone the moment the process exits, so writing it is the only chance you get.

That asymmetry — cheap now, impossible later — is unusual enough to be worth treating as a rule rather than a judgement. Most engineering decisions can be revisited; this one has a window that closes when the function returns.

How we proved it#

The question became answerable the day the field existed. Before it, "which path produced these" had no mechanical answer; afterwards, it is a count. That is not a small improvement in a workflow, it is a change in what kind of question it is.

The backfill is genuinely impossible, which is the strongest argument for stamping by default. Artifacts produced before the sidecar existed can only be classified by inference, and for same-shaped outputs from different paths, inference does not resolve. Those files are permanently unattributable.

Accepting that cleanly is better than pretending otherwise. I know the current state of everything produced since, I do not know the history of what came before, and writing that down as a boundary stops somebody later mistaking an inference about the old files for a fact about them.

What I cannot tell you is how often the field has been wrong. Nothing cross-checks a sidecar against the artifact it describes, so a mis-stamped file would be believed. I have no evidence of one and no mechanism that would produce evidence, which is the honest state.

If I wanted to close it, the move would be a periodic spot check: re-run the producing path on a sample and confirm the sidecar it writes matches what actually happened. That is a test of the stamping rather than of the artifacts, which is the right target, and it is the sort of check that is easy to describe and keeps not getting written.

What I have not solved: sidecars for things that are not files. A row in a table, a record created on a remote service, a value in a configuration store — each has the same question and no obvious place to put the answer that travels with it. The pattern needs a filesystem, and plenty of produced things do not live on one.

The closest equivalents I have seen are a column on the row and a tag on the remote object, and both work where they are available. What neither gives you is the property that makes sidecars pleasant: travelling automatically with copies. A row copied to another table does not bring its provenance unless somebody copies that too.

And a cost worth naming: it doubles the file count. Two hundred artifacts become four hundred entries in a directory listing, which is visually noisier, slower to scan by eye, and mildly annoying in a diff. I think the trade is clearly right and it is not free, and anybody who finds directory listings load-bearing will disagree.

The mitigation is a naming convention that sorts the sidecar immediately after its artifact, so a listing reads as pairs rather than as two interleaved populations. That is a small thing and it is the difference between a directory that is twice as long and one that is twice as confusing.

The thing that reframed it was realising how briefly the information exists. Inside the producing function, which path is running is completely known — it is the line of code that is executing. A microsecond later the process has moved on and the fact is unrecoverable by any means. Provenance is not something you can look up later; it is something you either write down at the one moment it is free, or lose.

Framing it that way also settles the usual objection, which is that this looks like speculative work for a question nobody has asked. It is not speculative; it is the only available ordering. The alternative is not "write it later if needed" — there is no later — it is "never know", and that is a different trade from the one it appears to be.

What this generalizes to#

Facts that are obvious during execution become unrecoverable immediately after it, and most of them are never written down because they were never in doubt at the time.

The pattern applies to anything derived. Which rule matched. Which configuration was in effect. Which version of an input was used. Which of several candidates was selected and why. All of these are completely determined while the code runs and completely absent from the result, and every one of them becomes the subject of an argument eventually.

So the useful question while writing anything that produces an output is what would somebody want to know about this in six months — because the answer is almost always something you have in hand right now and will never be able to reconstruct, and writing it beside the output costs a line.

The test I now apply is to imagine being asked, with no access to logs and no memory of the run, to prove how something was made. If the honest answer is that I could not, then the artifact is missing a fact it should be carrying — and the moment to fix that is while the code that knows is still running.