Sameer Singh

Sameer Singh, machine-learning and natural-language-processing researcher. An early-career researcher at the University of Washington at the time of LIME, later an associate professor at the University of California, Irvine. His work centres on the robustness, interpretability, and evaluation of NLP models, including adversarial testing and the AllenNLP Interpret toolkit.

Stake§

Professional and reputational — a co-author of a widely-cited method that established the model-agnostic explanation agenda. No commercial conflict at publication.

Singh is a co-author of LIME, the local-surrogate method that explains any classifier's predictions after the fact. His broader programme for this topic probes and tests models from the outside — the practical wing of post-hoc explanation — rather than opening their internals, which is the alternative the circuits work pursues.

Works in this corpus§

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

¶ excerpts

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model.

— "Why Should I Trust You?": Explaining the Predictions of Any Classifier (2016)

The framing that set the post-hoc [[concept:explainability|explanation]] agenda: the model stays a black box, but a local explanation of each prediction is offered as the basis for trust.

on Explainability

We propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally around the prediction.

— "Why Should I Trust You?": Explaining the Predictions of Any Classifier (2016)

LIME fits a simple, transparent model in the neighbourhood of a single prediction to approximate the black box there. The word "faithful" is the load the method is asked to carry — and the same word names the property that [[source:turpin-2023-language-models-dont-always-say-what-they-think|Turpin]] later finds chain-of-thought explanations lack.

on Explainability, Explanation faithfulness