Name the fault, and rule out the wrong fix

A call fails and the message says something about the input. The input is the thing the caller controls, so the caller edits it. An hour later the input has been rewritten four ways and nothing has changed.

The condition was a quota. The message never mentioned one, because the layer writing it could not see one.

The case this post is about#

One message: identity not found.

It is accurate. Something looked for a signing identity and did not find one. What a reader does next is go and check the certificate, because that is what the message names, and the certificate is fine — the default keychain had been unset, which the message cannot mention because that layer cannot see it.

Most of a day went into the wrong repository on the strength of three correct words.

This post follows that one message, because the fix is not more detail about what was sent. It is four specific things: what was observed, what this layer cannot see, which plausible repair will not help, and the one cheap test that discriminates.

The symptom#

Time spent on a hypothesis the error supplied. Not a wild guess — a focused, competent investigation of exactly the thing the message named, pursued properly, with bisection and simplification and careful notes. All of it directed at a component that was behaving correctly.

The second thing is that the message was not wrong, exactly. It described what the failing layer could observe, which was a request and a non-success. Everything it said was true. What it lacked was any statement about what it could not see, and a message that omits its own blind spot reads as complete.

The third is that the plausible wrong fix often appears to work. Simplifying the input occasionally succeeds, because a smaller unit of work sometimes fits where a larger one did not — so the false hypothesis gets confirmed by experiment, which is the most durable way to be wrong.

Experimental confirmation of a false theory is worse than no evidence at all, because it converts a hypothesis into a belief. From that point the investigation is no longer searching; it is elaborating, and every subsequent observation gets fitted into a frame that is already wrong.

The fourth is who pays. The author of the message knows the layer's limits perfectly well and would never be misled by it. Everybody else is reading it without that context, and the confident specificity that makes it feel helpful is exactly what makes it misleading.

That is a general hazard of writing for an audience you are a member of. The author reads their own message with the full model of the system behind it and judges it clear. Clarity to somebody holding the context is not the property that matters; the message is for people who do not have it.

Why the obvious fix failed#

Include more detail. More of the same kind of information. A longer description of the request does not become a statement about the service's state, and the extra detail increases confidence in the wrong direction.

Return the underlying error. Right when there is one. Frequently there is not: the layer saw a non-success with no explanation, and passing through an absence produces a message that says nothing at all — which is less misleading and also less useful.

The choice between those two is worth making deliberately rather than by default. A message that says nothing wastes nobody's afternoon and leaves them with no starting point; a message that guesses gives them a starting point in the wrong place. The third option — say what is known, state the limit, name a test — costs one more sentence than either and is the only one that helps.

Write better prose. Clarity applied to the wrong content. A beautifully phrased message about the request is still a message about the request, and nothing about the phrasing addresses the omission.

Add a troubleshooting link. Useful and downstream. By the time somebody follows a link they have already formed a theory from the message, and the link is read in the light of that theory rather than as a correction to it.

Which is a general property of anything that arrives after a theory has formed. The order in which information reaches somebody determines what it does: the same sentence read first reframes the problem, and read third gets interpreted to fit. That is an argument for putting the important part in the message rather than one click away.

The actual mechanism#

A message should name the condition, state what it is not, and point at the cheapest discriminating test.

condition   what the system believes happened, in its own terms
limits      what this layer cannot see, stated plainly
not-this    the plausible wrong repair, explicitly ruled out
test        the one cheap action that distinguishes the remaining cases

The third line is the one that is almost never written and does the most work. A message saying "this is not a problem with your request" removes an hour of competent investigation, and the author of the message is uniquely positioned to know it — they know what the layer can observe, so they know which theories it cannot support.

The fourth line is what converts a message from a description into a next step. "Run the minimal known-good request" is a sentence anybody can act on in thirty seconds, and it separates input-dependent from capacity-dependent faults completely. A message that names a test is worth more than one that names a cause, because the test is checkable and the cause is a claim.

The sub-case worth stating is honesty about uncertainty. A layer that does not know why something failed should say so, in those words, rather than offering the most common reason as though it were the finding. "The call did not succeed and this layer cannot determine why" is a genuinely better message than a confident guess, even though it feels like a worse one to write.

It feels worse because it reads as an admission, and admissions are uncomfortable to put in production output. But an honest absence sends somebody looking in the right places, and a confident guess sends them looking in one specific wrong place — which is strictly more expensive than looking anywhere.

The fix#

state      the observed condition, without inferring a cause
declare    what this layer cannot see
rule out   the repair somebody is about to attempt and should not
name       one cheap test that discriminates

Four fields, and the discriminating test is the one that turns a message into a next action:

type Diagnostic = {
  observed: string;        // what this layer SAW — no inference
  cannotSee: string[];     // what is outside this layer's knowledge
  notThis: string[];       // the repair about to be attempted that will not help
  discriminate: string;    // one cheap command that splits the remaining causes
};

export const fmt = (d: Diagnostic) => [
  `observed: ${d.observed}`,
  ...d.cannotSee.map((x) => `  this layer cannot see: ${x}`),
  ...d.notThis.map((x) => `  NOT this: ${x}`),
  `  to discriminate: ${d.discriminate}`,
].join("\n");

export const signingIdentityMissing = (team: string): Diagnostic => ({
  observed: `no signing identity for team ${team} in the default keychain`,
  cannotSee: ["whether the certificate is valid remotely", "whether membership lapsed"],
  notThis: ["re-running the build", "regenerating the provisioning profile"],
  discriminate: "list identities: if the keychain is empty, this is local state, not the certificate",
});

The message that replaced "identity not found":

$ build
observed: no signing identity for team ABCDE12345 in the default keychain
  this layer cannot see: whether the certificate is valid remotely
  this layer cannot see: whether membership lapsed
  NOT this: re-running the build
  NOT this: regenerating the provisioning profile
  to discriminate: list identities — if the keychain is empty, this is local
                   state, not the certificate

The NOT this lines are the expensive ones to omit. Regenerating a provisioning profile is twenty minutes and the obvious next move, and it is downstream of a keychain that has no identities in it at all.

The second line is a small discipline with an outsized effect. Every layer has a horizon, the author knows where it is, and the reader cannot. Writing it into the message transfers the one piece of context that makes everything else interpretable.

What makes this cheap is that none of it requires new information. The message already had everything it needed — it just also had an inference presented at the same confidence as an observation, and the entire improvement is separating those two.

Separating them is also what keeps the message correct when the system changes. An observation stays true; an inference is a guess about a system that will be refactored, and the inference is the half that quietly becomes wrong while the sentence stays in place.

How we proved it#

The wrong-fix hour is directly attributable. The message named the request; the investigation went into the request; the request was fine. That sequence is not an inference about human behaviour, it is what happened, and it would have been prevented by one sentence saying the request was not implicated.

The discriminating test exists and is trivial, which is what makes the fourth line practical rather than aspirational. A minimal known-good request, run during a failure, separates the two families of cause in thirty seconds. The only reason it was not run is that nothing suggested it.

What I cannot tell you is whether better messages would have helped sooner in general. I have one episode, improved one message, and have not had a comparable failure since — which is not evidence, because the absence of a repeat could be anything. The argument here is about mechanism, not about a measured reduction.

What I have not solved: messages from layers I do not own. Most of the misleading text in any system comes from libraries and services somebody else wrote, and no amount of discipline on my own messages changes what those say. The practical response is to wrap them — restating the condition with my own layer's limits attached — which adds indirection and is better than passing through a confident guess.

The wrapping does not need to be elaborate. Catching a third-party error and re-raising it with one added sentence — what my layer was attempting and what it cannot determine — keeps the original text available and stops it being the whole story. Most of the value is in the sentence, not in the mechanism.

And a limitation of ruling out the wrong fix: you have to know what it is. The third line depends on anticipating which repair somebody will reach for, which is easy after the first time it happens and guesswork before. In practice these sentences get written after an incident, which makes them a form of institutional memory rather than foresight.

Which is a perfectly good thing for them to be, as long as the loop is closed. Every hour lost to a plausible wrong fix is an opportunity to put one sentence into the message that prevents the next one — and that only happens if the response to a wasted hour is to edit the error text rather than to move on now that the real cause is known.

The thing that reframed it was noticing that an error message is read as evidence and written as a summary. The author is condensing what happened; the reader is building a theory of the world from it. Those two activities have completely different requirements, and almost every bad error message I have written was a reasonable summary that made a poor premise.

What this generalizes to#

Any report from a component is bounded by what that component can observe, and almost no report says where its boundary is.

The pattern runs through every layered system. A client reporting a timeout that was a DNS failure. A parser reporting malformed input on a truncated download. A build tool naming a missing dependency when the real problem is a path it never searched. In each case the reporter describes its own experience accurately and the reader takes it as a description of the system.

So the question worth asking of any diagnostic you write is what could this layer not possibly know — and then writing that down, because the reader has no way to work it out and will otherwise treat your summary as the whole picture.

The habit that follows is small: after writing any diagnostic, read it as somebody who knows nothing about the internals and ask what they would do next. If the answer is an action inside a component that is working correctly, the message is missing a sentence.