Cynthia Rudin

InterpretabilityExplainability

in Black box

Cynthia Rudin, professor of computer science, electrical and computer engineering, and statistical science at Duke University, where she directs the Interpretable Machine Learning Lab. Before the 2019 Nature Machine Intelligence paper she had built inherently transparent models for consequential settings — CORELS and related certifiably optimal rule lists, and sparse scoring systems for medical and criminal-justice use. She received the 2022 AAAI Squirrel AI Award for this line of work.

Stake§

Professional and ideological. The paper is a manifesto for the research programme she is identified with, and the claim that explaining black boxes is harmful raises the standing of the interpretable-models work she and her students produce. No commercial conflict; the advocacy is open.

Rudin's position for this topic is that high-stakes decisions should be made by models that are interpretable by construction rather than by fitting a post-hoc explanation to a black box, since an explanation is a separate model that can disagree with the thing it explains. The argument runs directly against the model-agnostic explanation of Ribeiro's LIME, and her running example is the COMPAS recidivism tool, where opacity shields a consequential system from scrutiny. She shares the distrust of post-hoc explanation with Lipton, who deflates the very vocabulary of "interpretable" that she builds on.

Works in this corpus§

their concepts on the territory
ExplainabilityExplainability InterpretabilityInterpretability

2 concepts in this scholar's webopen the full territory →

excerpts

trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practices and can potentially cause great harm to society.
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2019)

The paper's hinge is a distinction it insists on: an [[concept:explainability|explanation]] is a second model that approximates a black box after the fact, while an [[concept:interpretability|interpretable]] model is transparent by construction. Rudin's claim is that for consequential decisions the first is the wrong tool. The argument runs against the post-hoc programme that [[source:ribeiro-2016-why-should-i-trust-you-lime|LIME]] launched.

on Interpretability, Explainability