Zoom In: An Introduction to Circuits

Chris Olah led the interpretability ("Clarity") team at OpenAI when this was published, having earlier founded and edited Distill; in 2021 he became a co-founder of Anthropic. His co-authors — Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter — are the group that originated the "circuits" approach to neural-network interpretability. The paper is the opening article of the Circuits thread, the founding statement of a research programme its authors have pursued for years since.

It appeared in 2020 in Distill, the peer-reviewed web journal Olah had started, which published interactive, figure-heavy machine-learning articles to a standard of visual explanation that ordinary venues could not host. The argument is made by close looking: working in the convolutional vision model InceptionV1, the authors trace individual neurons that detect curves, edges, and object parts, and the weighted connections that assemble simple detectors into complex ones. From this they advance three claims — that features are the fundamental unit and correspond to directions in activation space, that features are wired into circuits, and that analogous features recur across different models and tasks.

The piece sits as a primary research report and as a manifesto. Its bet is the opposite of the one that would abandon opaque models: where Rudin argues that high-stakes systems should be interpretable by construction rather than explained after the fact, Olah's group argues that even a model not built to be transparent can be reverse-engineered from the inside. That bet runs against the demonstration in Jonas and Kording's microprocessor study, where standard analysis methods applied to a fully known chip fail to recover its logic — the circuits programme is, in part, a claim to have better methods than the ones that failed there. The vision-model circuits described here are scaled toward language models in Scaling Monosemanticity.

The authors' stake is reputational and intellectual, and total. This article states the thesis — that neural networks are not irreducible black boxes — that the same people then committed years of work to vindicating, at OpenAI and later at Anthropic. Distill was a nonprofit with no direct commercial interest at the time of writing, so the investment legible here is in the idea rather than in a product, though the later work at Anthropic carries the commercial dimension the early circuits papers did not.