Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

The authors were members of Anthropic's interpretability team; Adly Templeton is the lead author, and the senior author Chris Olah carries the "circuits" lineage forward from Zoom In four years earlier. The report extends a 2023 proof of concept, Towards Monosemanticity (Bricken, Templeton, Batson and colleagues), which had decomposed a one-layer toy transformer; here the same dictionary-learning technique is scaled to Claude 3 Sonnet, a deployed production model.

It was published in May 2024 on the Transformer Circuits Thread, Anthropic's own web publication. This is not a peer-reviewed venue: the report carries the institution's imprimatur rather than external review, a filter worth naming because the same company builds the model, runs the interpretability method, and publishes the result. The technique is the sparse autoencoder — a second network trained to express the model's dense internal activations as a sparse combination of many more "features," each of which the authors then attempt to label by finding what activates it. The headline demonstration accompanied a public release, "Golden Gate Claude," in which the bridge feature was clamped high and the model began steering every conversation toward, and identifying as, the Golden Gate Bridge.

The piece sits as a primary research report in the mechanistic interpretability line, the scaling-up of the circuits programme from vision models to a frontier language model. It sits in tension with another paper from the same research community: Turpin and colleagues showed that a model's stated reasoning is an unreliable guide to its computation, while this report claims that the model's internal features can be read and manipulated directly — opacity attacked from inside rather than taken on the model's word. Where Rudin would replace the black box, this work tries to make the existing one transparent.

The stake is commercial as well as reputational, and direct. Claude is Anthropic's flagship product, and a demonstration that its internals can be identified, named, and steered underwrites the company's positioning on safety and interpretability to investors, regulators, and customers — the people most served by the claim are also the people publishing it. The self-published status means the findings rest on the lab's own validation rather than independent replication, which a reader weighing the strength of the demonstration should hold against the vividness of the Golden Gate result.