Engineering

Your error message is an API, and changing it breaks someone

Somebody is grepping your log line, matching on your message, or alerting on its text.

MobileByteSensei 8 min read
practicesapi-designoperations

TL;DR

An error message looks like the most private thing in your codebase — a string for a human, changeable at will. It is not. By the time a service has been live for a year, its error text has accumulated consumers: a client matching on it, an alert rule grepping for it, a runbook, a support macro, a log-based metric.

So improving the copy breaks things, silently, in systems owned by other people. The fix is not to freeze your messages. It is to give every error a stable machine-readable code and let the prose be free — so there is something correct to depend on.

The case this post is about

An authentication endpoint with messages written for humans:

throw ApiException(
    status = 403,
    message = "Account locked. Try again in 30 minutes.",
)
HTTP/1.1 403 Forbidden
{ "error": "Account locked. Try again in 30 minutes." }

Clear, actionable, good copy. Shipped, and left alone for fourteen months.

Then a product request: make the message say the actual unlock time rather than a fixed thirty minutes.

message = "Account locked until ${formatTime(until)}."

A one-line change, reviewed by two people, with no API change — the status code, the shape, and the field name were all identical.

The symptom

Four breakages over the following week, none of them in the service that changed.

The mobile client stopped showing the right screen. It had been routing on the message text:

// in the app, shipped 11 months earlier
if (error.message.startsWith("Account locked")) showLockedScreen()
else showGenericError()

That one actually survived — startsWith still matched. The Android client used contains("Try again in 30 minutes") and did not.

A PagerDuty alert went silent. The rule counted log lines matching the old string, so locked_account_rate dropped to zero and nobody was paged for a real spike in lockouts:

alert: account_lockout_spike
  query: count_over_time({app="auth"} |= "Try again in 30 minutes" [5m]) > 50
  → 0 for six days

The support macro became wrong. It told users to wait thirty minutes, which was no longer what the system did.

And a BI dashboard's lockout funnel flatlined, because it bucketed by exact message string.

Nothing in the changed service's own tests failed. Every consumer was in a different repository, owned by a different team, and most of them were not aware they had a dependency.

Why the obvious fix failed

"Revert the message." We did, which restored the alert and left the product request unmet. It also taught exactly the wrong lesson — that error copy is frozen forever — which is how messages become stale and unhelpful.

"Tell the consumers to stop matching on text." Correct and insufficient: they match on text because there is nothing else to match on. An instruction without an alternative is not a fix.

"Add a version to the endpoint." The response shape never changed, so versioning the endpoint does not address it. This is the tell that the contract was never in the schema.

"Document that messages are unstable." We had, in a README nobody involved had read. A documented instability that the API gives consumers no way to avoid is a note, not a contract.

The actual mechanism

Three things combine.

A string in a response is machine-readable whether you intend it or not. Anything a program can parse, some program will parse — particularly when it is the only thing distinguishing two outcomes.

Consumers appear without telling you. An alert rule written during an incident at 2am, a client branch added in a hurry, a support macro written by someone outside engineering. None of these show up in your dependency graph, your API tests, or your code search.

Human-oriented and machine-oriented requirements conflict. Good copy is specific, contextual, localisable, and improved over time. A good machine contract is stable, opaque, and never changes meaning. One field cannot be both, so the field you have becomes whichever one someone depends on first.

An error response whose only distinguishing field is a prose message, consumed by a mobile client branching on a substring, an alert rule counting matching log lines, a support macro and a BI funnel bucketing on the exact text — none of them visible to the service's own tests, so improving the copy breaks four systems in other repositories

And the property that makes this worse than an ordinary undocumented coupling: the breakage is silent on both sides. A client branching on a substring does not throw when the substring stops matching — it takes the else:

// the mobile client, written during a release crunch
if (error.message.contains("insufficient funds")) {
    showTopUpSheet()
} else {
    showGenericError()        // ← where "Insufficient balance" now lands
}

Nobody is paged. The service's tests pass, because the service is behaving correctly. The client's tests pass, because they were written against the old string and nothing tells them the server changed. The only signal is a conversion metric drifting in a dashboard somebody looks at weekly, which is how a one-word copy improvement takes eleven days to connect to a drop in top-ups.

There is a second-order version that is worse again. Once one consumer depends on the text, the next person to improve the copy is blamed for the outage — so the team learns "do not touch error messages", and the strings calcify at whatever quality they happened to have. The cost is not the one incident; it is that every subsequent improvement to the thing users actually read now carries a risk nobody can estimate.

If the only way to tell two failures apart is the prose, the prose is the API. The question is not whether to version it but whether to provide something better to depend on.

The fix

Give every error a stable code. Let the message be free.

// the error model — code is the contract, message is the copy
@Serializable
data class ApiError(
    val code: String,          // STABLE. Never changes meaning. Never reused.
    val message: String,       // free to improve, localise, A/B test
    val details: Map<String, String> = emptyMap(),   // structured, typed facts
    val retryAfterSeconds: Int? = null,
)
HTTP/1.1 403 Forbidden
{
  "code": "ACCOUNT_LOCKED",
  "message": "Account locked until 14:30.",
  "details": { "lockedUntil": "2026-10-24T14:30:00Z" },
  "retryAfterSeconds": 1800
}

Four decisions:

code is an enum, not a free string. Defined in one place, exhaustively, so adding one is a visible change:

enum class ErrorCode {
    ACCOUNT_LOCKED, INVALID_CREDENTIALS, TOKEN_EXPIRED, RATE_LIMITED, …
}

Structured facts go in details, not in the sentence. lockedUntil as an ISO timestamp is what a client needs to render its own copy in its own locale. Parsing "14:30" out of a sentence is what consumers do when you force them to.

Codes are never reused or re-meaninged. Retiring one means retiring the string. This is the discipline that makes it a contract rather than another unstable field.

And the message becomes genuinely free — which is the payoff. Copy can be improved, localised, and tested without a cross-team coordination problem.

Then migrate the consumers, which is the actual work:

// client, after
when (error.code) {
    "ACCOUNT_LOCKED" -> showLockedScreen(until = error.details["lockedUntil"]?.let(Instant::parse))
    "TOKEN_EXPIRED"  -> refreshAndRetry()
    else             -> showGenericError(error.message)   // message only for DISPLAY
}
alert: account_lockout_spike
  query: count_over_time({app="auth"} | json | code="ACCOUNT_LOCKED" [5m]) > 50

And a guard that makes message changes safe going forward:

// a test that pins the CODES, deliberately not the messages
@Test
fun errorCodesAreStable() {
    val expected = setOf("ACCOUNT_LOCKED", "INVALID_CREDENTIALS", "TOKEN_EXPIRED", "RATE_LIMITED")
    assertEquals(expected, ErrorCode.entries.map { it.name }.toSet())
}

A removed or renamed code fails CI. A reworded message does not — which is exactly the asymmetry you want, and it documents the contract in the place people actually look.

How we proved it

That consumers existed, found by searching outside our own repository. This was the step that changed the conversation from "someone might depend on this" to a list:

$ gh search code 'Try again in 30 minutes' --owner ourorg
  mobile-android/src/.../ErrorMapper.kt:41
  mobile-ios/Sources/.../APIError.swift:88
  observability/alerts/auth.yaml:12
  analytics/dbt/models/lockout_funnel.sql:30
  support-tooling/macros/locked.json:4

Five consumers, four teams, none in our dependency graph. Running that search before the change would have cost two minutes.

That the alert had been blind. Reconstructed from the metric's own history:

locked_account_rate (log-derived)
  Oct 10–16   avg 34/5min
  Oct 17      message changed
  Oct 17–23   0/5min            ← six days unmonitored
  Oct 23      alert rule fixed
  Oct 23–24   avg 41/5min       ← the rate had gone UP during the blind period

The last line is the part that mattered: lockouts increased while the alert was silent, so the message change cost us the detection of a real regression.

And that the new contract survives a message change — the regression test for the whole incident:

@Test
fun clientRoutesOnCodeNotMessage() {
    val a = ApiError("ACCOUNT_LOCKED", "Account locked until 14:30.")
    val b = ApiError("ACCOUNT_LOCKED", "Your account is temporarily unavailable.")
    assertEquals(route(a), route(b))      // same screen, different copy
}
old client (message matching):  FAILED
new client (code matching):     PASSED

One limit worth stating: migrating five consumers across four teams took three weeks, and for part of that period the service emitted both the old message format and the new code so that unmigrated consumers kept working. Dual-emitting was the thing that made the migration possible, and it is the same expand-contract shape as a schema change.

What this generalizes to

Anything parseable becomes an interface. Log formats, exit codes, file names, CLI output, exception messages, HTML structure. "For humans" is a statement of intent, not a technical property, and intent does not stop a program from depending on it.

Provide the stable thing, or your unstable thing becomes the stable thing. Consumers match on prose because nothing better is offered. A code, a type, a header, an enum — give them the thing you are willing to keep, and the pressure moves off the thing you are not.

Your dependency graph does not include your consumers. Alerts, dashboards, runbooks, support tooling and other teams' clients are real dependencies that no build tool knows about. An org-wide code search is the cheapest way to find them, and it takes minutes.

Separate the display field from the decision field. One string serving both a human and a program guarantees that improving it for one breaks it for the other. Two fields cost almost nothing and remove the conflict permanently.

Keep reading

This is part of 100 practices of senior engineers — the habits that separate five years of experience from one year five times. Follow MobileByteSensei for the rest.

One post a week, no digest

New writing when it lands. No roundups, no course pitch, no "10 tips".

Follow by RSS