Finale Doshi-Velez

Interpretability

in Black box

Finale Doshi-Velez, professor at Harvard's School of Engineering and Applied Sciences, working on interpretability and machine learning in healthcare. With Been Kim she wrote Towards A Rigorous Science of Interpretable Machine Learning (2017), one of the field's standard reference points on how to define and evaluate interpretability.

Stake§

Academic and reputational — being the citation reached for when noting that interpretability lacks a definition is itself a substantial professional return. No commercial interest in the argument.

Doshi-Velez's contribution to this topic is to make "interpretability" measurable rather than assumed. The paper proposes a three-part scheme for evaluating it — application-grounded (real users on real tasks), human-grounded (simplified tasks with lay people), and functionally-grounded (a formal proxy with no humans in the loop) — which gave the field a shared way to ask whether an interpretability claim had been tested at all. It runs in parallel with Lipton's deflation of the same term.

Works in this corpus§

their concepts on the territory
InterpretabilityInterpretability

1 concept in this scholar's webopen the full territory →

excerpts

Unfortunately, there is little consensus on what interpretability in machine learning is and how to evaluate it for benchmarking.
Towards A Rigorous Science of Interpretable Machine Learning (2017)

The complaint that organises the paper, made the same year as [[source:lipton-2018-mythos-model-interpretability|Lipton's]]: the field uses [[concept:interpretability|interpretability]] as a goal without a shared definition or a way to measure whether a method has delivered it. The paper's contribution is a taxonomy of evaluation rather than a new method.

on Interpretability