Index Errans

person in Black box

Adly Templeton

Mechanistic interpretability

in Black box

Adly Templeton, interpretability researcher at Anthropic and lead author of Scaling Monosemanticity (2024), the work behind the "Golden Gate Claude" demonstration. The paper applies sparse autoencoders to Claude 3 Sonnet to extract interpretable features from a deployed model.

Stake§

Commercial and reputational — Claude is Anthropic's product, and a demonstration that its internals can be identified and steered underwrites the company's positioning on safety and interpretability. The report is self-published, without external peer review.

Templeton led the scaling of mechanistic interpretability from toy models to a frontier system for this topic. Scaling Monosemanticity extracts millions of features with sparse autoencoders and shows control as well as reading: clamping the Golden Gate Bridge feature high makes the model steer every exchange toward, and identify as, the bridge. The work continues the circuits line of Olah and sits in tension with Turpin's finding that the same kind of model's stated reasoning cannot be taken at face value.

Works in this corpus§

their concepts on the territory
Mechanistic interpretabilityMechanistic interpretability

1 concept in this scholar's webopen the full territory →

excerpts

For instance, we see that clamping the Golden Gate Bridge feature 34M/31164353 to 10× its maximum activation value induces thematically-related model behavior. In this example, the model starts to self-identify as the Golden Gate Bridge!
— Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (2024)

The "Golden Gate Claude" demonstration: a single extracted feature can be turned up, and the model's behaviour changes in a way that names the feature's meaning. The claim is that the feature is a handle on the concept rather than merely a correlate of it — opening and steering the box at once.

on Mechanistic interpretability