Index Errans

person in Black box

Trenton Bricken

Mechanistic interpretability

in Black box

Trenton Bricken, interpretability researcher at Anthropic. Lead author of Towards Monosemanticity (October 2023), the one-layer-transformer proof of concept that Scaling Monosemanticity scaled to Claude 3 Sonnet, and a co-author of the 2024 work. His background is in computational neuroscience, including work on attention and associative memory.

Stake§

Commercial and reputational, as a member of the Anthropic interpretability team whose results support the company's safety positioning; the work is self-published rather than peer-reviewed.

Bricken's contribution to this topic is the dictionary-learning method at the base of Anthropic's mechanistic interpretability: training a sparse autoencoder to express a model's dense activations as a sparse combination of many more, individually interpretable features. Towards Monosemanticity established the technique on a single-layer model, and the 2024 paper he co-authored carried it to a production system.

Works in this corpus§

their concepts on the territory
Mechanistic interpretabilityMechanistic interpretability

1 concept in this scholar's webopen the full territory →

excerpts

For instance, we see that clamping the Golden Gate Bridge feature 34M/31164353 to 10× its maximum activation value induces thematically-related model behavior. In this example, the model starts to self-identify as the Golden Gate Bridge!
— Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (2024)

The "Golden Gate Claude" demonstration: a single extracted feature can be turned up, and the model's behaviour changes in a way that names the feature's meaning. The claim is that the feature is a handle on the concept rather than merely a correlate of it — opening and steering the box at once.

on Mechanistic interpretability