Index Errans

person in Black box

Eric Jonas

Black-box method

in Black box

Eric Jonas, computational scientist. In the Department of Electrical Engineering and Computer Science at the University of California, Berkeley when "Could a Neuroscientist Understand a Microprocessor?" (PLOS Computational Biology, 2017) appeared, later on the computer-science faculty at the University of Chicago. His work spans machine learning, scientific computing, and measurement.

Stake§

Reputational and methodological — a deliberately provocative field check aimed at neuroscience's own analysis toolkit, using a system whose ground truth is fully known. No commercial or ideological interest.

Jonas, with Konrad Kording, ran the black-box method on a system where the answer is known: a MOS 6502 microprocessor running classic games, analysed with the standard toolkit of systems neuroscience. The methods surface plausible-looking structure without recovering how the chip computes — a caution that cuts two ways for this topic. It humbles the reading of a biological brain, and it warns that machine interpretability methods can feel like understanding without delivering it, the standing challenge to Olah's circuits programme.

Works in this corpus§

their concepts on the territory
Black-box methodBlack-box method

1 concept in this scholar's webopen the full territory →

excerpts

We show that the approaches reveal interesting structure in the data but do not meaningfully describe the hierarchy of information processing in the microprocessor.
— Could a Neuroscientist Understand a Microprocessor? (2017)

The test: take a system whose every wire is known — the MOS 6502 running Donkey Kong, Space Invaders, and Pitfall — and run standard neuroscience analyses on it. The methods produce results that look like findings but do not recover the logic that is sitting right there in the schematic.

on Black-box method

Ultimately, the problem is not that neuroscientists could not understand a microprocessor, the problem is that they would not understand it given the approaches they are currently taking.
— Could a Neuroscientist Understand a Microprocessor? (2017)

The problem the authors locate is in the methods rather than the target: a toolkit that cannot recover the logic of a chip with a known answer gives reason to doubt its verdicts on the brain, where there is no answer key.

on Interpretability

⤺ mentioned in

articles