The dependency with one maintainer, and the week it went away
Bus factor is a property of your supply chain, not just your team.
operations
20 posts
Rotating the key was the easy part
A committed secret is public the moment it is pushed. Deleting the line changes nothing.
The health check that returned 200 while the service was useless
Proving the process is running keeps a broken instance in the load balancer.
The rollout with no off switch, discovered at 2am
A flag that gates the new path but not the data migration is not a kill switch.
The config change that was really a deploy, and had none of the safeguards
Config bypasses code review, CI, canaries and rollback. It changes behaviour just as much.
Logs are for the questions you have not thought of yet
A metric answers a question you already had. The difference is cardinality.
The dashboard nobody looked at, and the alert that fired every day
An alert that fires daily has trained everyone to ignore it, which is worse than no alert.
Your error message is an API, and changing it breaks someone
Somebody is grepping your log line, matching on your message, or alerting on its text.
The migration that could not be rolled back, and nobody noticed until it mattered
A column drop is irreversible the moment it runs. Expand-contract exists because deploys and migrations do not move together.
The retry that turned a blip into an outage
Synchronised retries from every client are a self-inflicted thundering herd. Jitter is the mechanism, not a detail.
A feature flag read on four platforms needs one source of truth and four caches
Flags that differ per platform at the same moment produce bug reports nobody can reproduce.
The third time this filled up, I diagnosed it as an outage
A consumable with no recorded lifetime is indistinguishable from a service that is down.