Index Errans

person in Black box

Been Kim

InterpretabilityMechanistic interpretability

in Black box

Been Kim, interpretability researcher, a staff research scientist at Google Brain (later Google DeepMind). With Finale Doshi-Velez she wrote Towards A Rigorous Science of Interpretable Machine Learning (2017); she is known for TCAV (testing with concept activation vectors) and SmoothGrad.

Stake§

Academic and reputational, agenda-setting for a subfield she helped define; no commercial conflict in the position paper.

Kim's work for this topic sits on both sides of the open-the-box question. The 2017 position paper argues that interpretability needs rigorous evaluation; her TCAV method tests whether a human-chosen concept — "stripes," say — influences a model's prediction, an attempt to read a network in terms a person can name that sits alongside the feature-level mechanistic work of Olah.

Works in this corpus§

their concepts on the territory
InterpretabilityInterpretability Mechanistic interpretabilityMechanistic interpretability

2 concepts in this scholar's webopen the full territory →

excerpts

Unfortunately, there is little consensus on what interpretability in machine learning is and how to evaluate it for benchmarking.
— Towards A Rigorous Science of Interpretable Machine Learning (2017)

The complaint that organises the paper, made the same year as [[source:lipton-2018-mythos-model-interpretability|Lipton's]]: the field uses [[concept:interpretability|interpretability]] as a goal without a shared definition or a way to measure whether a method has delivered it. The paper's contribution is a taxonomy of evaluation rather than a new method.

on Interpretability