Index Errans

person in Black box

Zachary C. Lipton

Interpretability

in Black box

Zachary C. Lipton, machine-learning researcher. A PhD student in the machine-learning group at the University of California, San Diego when The Mythos of Model Interpretability first appeared in 2016, and later an associate professor at Carnegie Mellon, where he runs the Approximately Correct Machine Intelligence lab. He is known as a critic of loose terminology and hype in the field, writing the blog Approximately Correct.

Stake§

Reputational and intellectual — a contrarian methodological intervention of the kind that establishes a young researcher as the field's skeptic. The argument cuts against commercial explainability tooling rather than toward it.

Lipton's contribution to this topic is to show that "interpretability" is invoked without an agreed meaning, then to sort the competing senses — the desiderata people want (trust, causality, transferability, informativeness, fairness) from the model properties proposed to deliver them (transparency versus post-hoc explanation). His aside that humans exhibit none of the transparency the field demands of models, and that the processes by which people decide and the processes by which they explain may differ, is the confabulation finding of Nisbett and Wilson arriving inside a machine-learning paper.

Works in this corpus§

their concepts on the territory
InterpretabilityInterpretability

1 concept in this scholar's webopen the full territory →

excerpts

And yet the task of interpretation appears underspecified. Papers provide diverse and sometimes non-overlapping motivations for interpretability, and offer myriad notions of what attributes render models interpretable. Despite this ambiguity, many papers proclaim interpretability axiomatically, absent further explanation.
— The Mythos of Model Interpretability (2018)

Lipton's opening charge: the field demands [[concept:interpretability|interpretability]] without agreeing on what the word names. The paper then sorts the competing meanings into desiderata (trust, causality, transferability, informativeness, fairness) and properties (transparency versus post-hoc explanation).

on Interpretability

To the extent that we might consider humans to be interpretable, it is this sort of interpretability that applies. For all we know, the processes by which we humans make decisions and those by which we explain them may be distinct.
— The Mythos of Model Interpretability (2018)

The line that puts the human and the machine on the same footing: the explanations people give may be a separate process from the decisions they explain. This is the [[concept:confabulation|confabulation]] finding of [[source:nisbett-wilson-1977-telling-more-than-we-can-know|Nisbett and Wilson]] arriving inside a machine-learning paper, and the symmetry [[source:turpin-2023-language-models-dont-always-say-what-they-think|Turpin]] later demonstrates for language models.

on Interpretability, Confabulation

⤺ mentioned in

articles