"Why Should I Trust You?": Explaining the Predictions of Any Classifier

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin were all at the University of Washington when this appeared: Ribeiro a doctoral student, Singh an early-career researcher, and Guestrin the Amazon Professor of Machine Learning, who soon moved to Apple through the acquisition of his company Turi. The paper introduced LIME (Local Interpretable Model-agnostic Explanations), and it became one of the most-cited explanation methods in the field, with SHAP and a large secondary literature following from the model-agnostic framing it set out.

The work was published in 2016 in the proceedings of KDD, the principal data-mining conference, a competitive peer-reviewed venue. Its proposal is technical and deliberately general: rather than open any particular model, LIME treats every classifier as a black box and learns a simple, interpretable surrogate — a sparse linear model — in the local neighbourhood of a single prediction, so that each individual output comes with an account of which features pushed it. The paper pairs the method with human-subject experiments on whether such explanations help people decide to trust a model and spot when it is right for the wrong reasons.

The piece sits as a primary methods contribution, the founding statement of the post-hoc, model-agnostic pole of the debate. It is the position Rudin argues against directly three years later, on the ground that an explanation which can disagree with the model it explains is the wrong instrument for high-stakes use. Its central promise — explanations that are "faithful" to the underlying model — is the property Lipton frames as underspecified and that Turpin and colleagues later show can fail outright for chain-of-thought reasoning. Its legal cousin is Wachter, Mittelstadt and Russell's counterfactual explanations, which likewise aim to give an account of a decision "without opening the black box."

The authors' stake was professional and reputational: the paper launched a research agenda and, with it, Ribeiro's career, and it had an interest in establishing that explaining black boxes post hoc is both feasible and trustworthy — the opposite bet from the one Rudin would place. Guestrin's industry position, at Turi and then Apple, sat adjacent to a commercial market for explainability tooling, though the KDD paper itself carries no product.