Towards A Rigorous Science of Interpretable Machine Learning

Finale Doshi-Velez was a faculty member at Harvard's School of Engineering and Applied Sciences, working on interpretability and machine learning in healthcare; Been Kim was a research scientist at Google Brain, known for TCAV (testing with concept activation vectors) and SmoothGrad. The two are among the most-cited names in the interpretability subfield, and this short paper is one of its standard reference points. The arXiv version prints only an equal-contribution note rather than institutional affiliations, so the Harvard and Google Brain attributions come from the authors' own pages, not the preprint masthead.

It was posted to arXiv in February 2017 as a position paper and never sent through a journal; its influence comes from being early, brief, and widely assigned rather than from a peer-review stamp. The argument is that "interpretability" is invoked as a goal without rigour, and the constructive proposal is a three-part scheme for evaluating it: application-grounded (real users on real tasks), human-grounded (simplified tasks with lay humans), and functionally-grounded (a formal proxy with no humans in the loop). The taxonomy gave the field a shared vocabulary for arguing about whether an interpretability claim had been tested at all.

The piece sits as a framing and agenda-setting paper, secondary to the methods it would discipline. It runs in parallel with Lipton's Mythos of Model Interpretability — both published around 2016–2017, both arguing that the term is underspecified — though where Lipton dissolves the concept into competing desiderata, Doshi-Velez and Kim try to make it measurable. It points toward Miller's later case that the human-grounded end of evaluation should draw on the existing cognitive science of how people explain.

The authors' stake is academic and reputational. The paper is an attempt to set terms for a subfield they helped define, and being the citation everyone reaches for when noting that interpretability lacks a definition is itself a substantial professional return. There is no commercial interest in the argument; both authors went on to build interpretability methods that the evaluation scheme proposed here would be used to judge.