Counterfactual explanation

counterfactual recourse
the idea

A way of explaining an automated decision by naming the smallest change to the inputs that would have flipped the outcome — for instance, that an applicant earning a little more would have been approved. Rather than opening up how the system works inside, it tells the affected person what they would have needed to do differently. It trades insight into the mechanism for something the subject can act on.

An account of an automated decision stated as the smallest change to the inputs that would have produced a different outcome — "you were denied because your income was £30,000; at £45,000 you would have been approved." Proposed by Wachter, Mittelstadt and Russell (2017/2018) as a remedy that gives a person recourse without disclosing the model's internals.

The counterfactual explanation is explainability reframed as recourse. Wachter, Mittelstadt and Russell drop the demand to see inside the model and ask instead what the subject needs in order to act, judging an explanation by the action it enables rather than the mechanism it reveals. The technical content — the smallest change to the world that flips the decision — is Chris Russell's; the legal setting is the debate over whether Europe's General Data Protection Regulation contains a right to explanation, which the same authors argued elsewhere it does not.

The method shares LIME's stance of explaining a model from outside, and so inherits Rudin's objection that an external account can diverge from the model it describes. Its appeal is that it answers a narrower, checkable question — what would have had to be different — and leaves the opacity itself in place.

Discussed in§

you are here in the territory

2 direct·3 two hops·18 further·46 off-graphopen the full territory →