Where a secret lives should never be a decision

A signing keystore has to be on disk somewhere before a build can use it. So something writes it to a path, and something else reads it from a path, and those two things are usually written months apart by people solving different problems.

They agree until the day one of them moves.

The case this post is about#

One keystore, and the four places that each decided for themselves where keystores live:

$ ./gradlew verifySecretsLayout
✗ deployment/fastlane/Fastfile:22  reads SECRETS_LAYOUT.yaml directly
    use: $(bash core/scripts/secrets-resolve-path.sh <alias>)
FAILED — 1 direct registry read
$ bash core/scripts/secrets-resolve-path.sh acme-upload-keystore
keystores/upload.jks

Fastlane knew the path, the Gradle signing config knew the path, a CI workflow knew the path, and the dev-build script knew the path. All four agreed, which is why nothing surfaced for months.

Then one of them changed. This post follows that keystore, because four consumers holding the same literal is not agreement — it is four copies of a fact, and the disagreement they are heading for surfaces during a release.

The symptom#

Count the places that know where a keystore lives. The script that materialises it from the vault. The build file that signs with it. The CI workflow that provisions it. The local dev setup. The documentation that tells a new contributor what to expect. The ignore file that keeps it out of version control.

That is six independent copies of one fact, in five languages, across at least two repositories. Each one is correct. None of them is the source.

The second thing is when the disagreement shows up. Secrets are materialised rarely — on a fresh machine, in CI, at release time. Day-to-day development never touches the path, because the file is already sitting where it was put the first time. So a mismatch introduced in March is discovered in June, by whoever is trying to ship, and the gap between the change and the symptom is long enough that nobody connects them.

The third is that the error actively misleads you. A build that cannot find a keystore at the path it expects reports, in effect, no keystore. That reads as not provisioned — so the natural next move is to go and provision it: pull from the vault, re-run the setup, maybe regenerate. All of which is work aimed at a problem you do not have, because the file exists, intact, forty characters away.

The fourth is that it is not one decision, it is a grid. The right location depends on the ecosystem — a Kotlin Multiplatform build wants properties, a Node service wants JSON, a Python service wants TOML — and on the category of secret: a keystore is a binary file, an API token is an environment variable, a service-account credential is a JSON document with its own conventions. Fifteen or so combinations, each with a different right answer, and every consumer is working that two-dimensional problem out independently.

Why the obvious fix failed#

Write the paths down. The first thing anybody does, and it fails for a structural reason rather than a human one: documentation is not in the call path. The build does not read the document. Two artifacts that are supposed to agree, where only one of them is executed, will diverge — and the one that diverges silently is always the document, because nothing fails when it is wrong.

Define a shared constant. Right instinct, wrong reach. A constant works beautifully inside one codebase and the consumers here are in different repositories and different languages. There is no import that a Gradle file, a shell script, a GitHub workflow and a Python service can all share.

Put the path in an environment variable. This feels like indirection and is mostly relocation: the literal still exists, it is now in CI configuration or a shell profile instead of in code, and it is less visible rather than more. It also has to be set in every context, and the context where it is missing is the one nobody tests — which converts a wrong-path error into an empty-path error, strictly worse because an empty path often resolves to somewhere that exists.

Put everything in one directory. The tempting simplification, and it ignores why the paths differ in the first place. They are not arbitrary. A properties file has to be where the build system looks for properties; a service-account JSON has to be where the SDK expects credentials. The variation is imposed by the ecosystems, and a scheme that flattens it just moves the problem into a pile of symlinks.

The actual mechanism#

Two parts, and the second is the one that is easy to skip and does all the work.

One registry, keyed by the pair. A single table mapping (ecosystem, category) to the canonical path. That is the whole content of the thing — fifteen or so rows, each one a fact that was previously implicit in six places.

One resolver, and direct reads of the registry are banned. This is the half that matters. A registry alone just relocates the duplication: instead of six hardcoded paths you now have six pieces of code that each parse the registry and each make their own assumptions about its shape. The moment the registry gains a field or changes nesting, five of them break and one of them silently returns something wrong.

So consumers never read the registry. They call a resolver — resolve <alias> in, a path out — and the registry has exactly one reader. That makes the registry's format a private implementation detail rather than a public interface, which is what allows it to change at all.

The sub-case that determines whether this works in practice: the resolver has to be reachable from every ecosystem, and that rules out the natural choice. A library is the obvious shape and it is wrong here, because a library exists in one language and the consumers are a Gradle file, a shell script, a workflow and two services. What crosses all of those is a process: a small executable with a stable command line and a path on stdout. Unfashionable, and it is the only interface all five can call.

There is a second lookup hiding in this that is worth pulling out, because it is where the scheme either stays honest or quietly rots. Consumers do not ask for (kmp, android_keystore) — they ask for a named secret, and something has to turn that name into the pair. So the alias is itself an interface, and an alias that does not say what it is produces a registry whose rows are correct and whose keys are guesswork.

Which makes naming part of the mechanism rather than a style question. An alias has to carry its scope — whether the value is shared across a whole organisation or belongs to one application — because that scope determines which project can legitimately ask for it. Two applications sharing an organisation-wide publishing credential is correct; one application reaching for another's signing key is not, and the only thing standing between those two cases is whether the name made the distinction visible. Get that wrong and the resolver cheerfully hands over a path it should have refused.

There is also a decision about failure. A resolver that cannot find a mapping must exit non-zero with the pair it was asked about, not fall back to a default. A default here is a path that exists, is wrong, and produces a confusing downstream error — which is the exact failure mode the whole scheme is meant to remove.

The fix#

registry:  (ecosystem, category) → canonical path     ← the only place the fact lives
resolver:  one executable, path on stdout             ← the only reader of the registry
consumers: call the resolver                          ← no literals, no registry parsing
gate:      fail the build on a direct registry read   ← what keeps the above true

The gate is the part that looks like bureaucracy and is actually load-bearing. Without it, the first person in a hurry greps the registry directly — reasonably, because it is right there and parsing one line is faster than shelling out — and the single-reader property is gone, silently, in a commit that

looks like a small cleanup.

The resolver is small enough that the discipline costs nothing — which is the point, because a resolver people find slow is a resolver people route around:

# secrets-resolve-path.sh — the ONLY reader of the layout registry.
set -euo pipefail
alias_id="${1:?usage: secrets-resolve-path.sh <alias>}"

stack=$(yq -r ".aliases.\"$alias_id\".tech_stack // \"\"" "$REGISTRY")
category=$(yq -r ".aliases.\"$alias_id\".category // \"\"" "$REGISTRY")
[ -n "$stack" ] && [ -n "$category" ] || {
  echo "unknown alias: $alias_id — add a row to $REGISTRY" >&2; exit 3; }

path=$(yq -r ".layout.\"$stack\".\"$category\" // \"\"" "$REGISTRY")
[ -n "$path" ] || {
  echo "no canonical path for ($stack, $category)" >&2
  echo "  add one to $REGISTRY rather than inventing a path at the call site" >&2
  exit 4; }

printf '%s\n' "$path"

And the gate that keeps it the only reader — a grep for anyone parsing the registry directly:

$ ./gradlew verifySecretsLayout
✗ deployment/fastlane/Fastfile:22  reads SECRETS_LAYOUT.yaml directly
    use: $(bash core/scripts/secrets-resolve-path.sh <alias>)
FAILED — 1 direct registry read

$ bash core/scripts/secrets-resolve-path.sh acme-upload-keystore
keystores/upload.jks

Exit 4 is the interesting one. An unmapped (stack, category) is a loud failure naming the file to edit — not a default path that happens to work on one machine and writes a credential somewhere nobody is looking. looks like a small cleanup.

Per-project manifests then become a pure list of which aliases a project needs, with no paths in them at all. That is the test for whether the separation is real: if a manifest still contains a path, the project is still making the decision.

How we proved it#

The registry exists and is small. Fifteen (ecosystem, category) rows cover every secret shape in use here: environment variables split by client and server, Android keystores and their properties, Firebase configuration per platform, the Play Store and App Store credentials, Maven and GPG signing material, OAuth client secrets, service accounts. The fact that it fits in fifteen rows is itself evidence that the variation was never as large as six scattered copies made it look.

Consumers call the resolver. The Fastlane configuration, the CI workflows, the materialisation script and the dev build all go through one executable, and a check fails the build on a direct registry read.

What I cannot tell you is how many incidents this prevented. I do not have a before-and-after count, because the before state was not instrumented — it was six places that happened to agree. The honest version is that this fixed a class of problem I could demonstrate was possible and had hit, and I am not able to put a frequency on it.

What I have not solved: the ignore file still has a literal. Version-control ignore patterns cannot call a resolver — they are matched by a tool that will never shell out — so the one place a secret path is still hardcoded is the file whose job is keeping secrets out of the repository. That is an uncomfortable place for the exception to land, and the mitigation is a broad pattern rather than an exact path, which is less precise than I would like.

And a limitation in the design itself: the resolver resolves location, not presence. It will happily return the canonical path for a secret that was never materialised, and the caller then gets a missing-file error at that path. That is a better error than before — it names the right place — but the resolver cannot tell you whether the thing is there, and more than once I have read its output as confirmation that it was.

What this generalizes to#

Any fact derived from two or more variables, consumed in more than one language, needs one implementation with a process boundary — not a constant, not a document, and not a table that everyone parses for themselves.

The tell is a value that several components compute independently and that they all currently agree on. Agreement is not evidence of a shared source; it is usually evidence that nothing has changed yet. Ask what happens on the day one of them changes, and whether anything would fail at that moment or six weeks later during a release.

And when you do centralise it, centralise the reading as well as the storing. A single source of truth with many parsers is not a single source of truth — it is one file and several interpretations of it, which is the same problem with better branding.