Black box

active 1 article 10 concepts 28 people 18 sources

The problem of a system known only from the outside — its inputs and outputs visible, the conversion between them closed to inspection — and the argument over whether the demand to open it should fall on machines, on minds, or on both. The method is old: W. Ross Ashby's 1956 cybernetics gave the black box its general statement and applied it in the same paragraphs to a sealed circuit and to a damaged brain. The modern machine form runs through the interpretability literature — the case for replacing opaque models (Rudin), the charge that "interpretability" is itself underspecified and that humans are no more transparent (Lipton), the post-hoc explanation methods (Ribeiro's LIME; Wachter's counterfactuals), and the mechanistic programme that claims to reverse-engineer a network feature by feature (Olah's circuits; Anthropic's Golden Gate Claude). The human form is the confabulation literature — Nisbett and Wilson on the absence of introspective access, Gazzaniga's left-hemisphere interpreter, Haidt's post-hoc moral reasoning. The two halves meet where a model's stated reasoning turns out not to be its real reason (Turpin), and where standard analysis methods fail on a chip whose wiring is fully known (Jonas and Kording). The stakes are drawn by Pasquale on the politics of algorithmic secrecy and by Dennett, whose intentional stance asks why a machine should owe a legibility that no mind has ever provided.

view the territory →

27 dated entities · 1903–2026 · sources above, people below

¶ articles

The black box and the other black box

A plain-language tour of the central tension: we ask AI to explain itself, then remember we were never able to explain ourselves either. It runs from Ashby's 1956 black box through the interpretability fight (Rudin, Lipton, Olah), into the human confabulation literature (Nisbett and Wilson, Gazzaniga's interpreter, Haidt), to the twist that a language model confabulates the same way — and the sharper point that we trained it to, through RLHF. Along the way it separates two kinds of opacity (a brain with no blueprint, a network with a blueprint too big to read), gives the mechanistic-interpretability camp its strongest reply to the microprocessor result, and ends on Dennett's intentional stance — with a caveat for where that stance snaps on a mind shaped nothing like ours. An AI-synthesized draft, in a deliberately chill register, for the curator to revise.