AI interpretability: a new challenge to the superposition hypothesis

Understanding why an AI model gives a particular answer requires examining its internal calculations. This field is called mechanistic interpretability. A neuron here is a numerical unit in an artificial network, rather than a brain cell. The current dispute is whether individual neurons offer clearer explanations than researchers previously assumed.

The direct-neuron hypothesis

Goodfire’s Nick Cammarata proposes studying native neurons and circuits directly. He reinterprets the cat/car neuron as a shared V-shape detector. His “recursive generalization” describes related branching, illustrated by an “iy” detector extending to “diy” and “do it yourself.”

His preliminary Qwen3-32B audit assigns about 90% of positive/negative neuron poles one principal role, around 7% two related roles, and leaves 2–4% unresolved. These estimates depend on definitions and activation thresholds. At publication, the detailed companion paper was still forthcoming.

Why superposition became influential

Anthropic’s September 2022 toy-model paper demonstrated that small networks trained on sparse synthetic features can represent more features than available dimensions. Compression introduces interference, which nonlinear operations can reduce. Its contribution was a controlled demonstration, not proof that every real model works this way. The authors explicitly cautioned that generalizing to practical networks remained uncertain.

The dictionary-learning alternative

In October 2023, Anthropic used sparse autoencoders to decompose a one-layer transformer’s activations. One central analysis represented a 512-neuron layer’s activations using 4,096 learned features. The researchers reported interpretable patterns and interventions that changed generated text. They did not claim every feature interpretation was complete. This established a concrete alternative to treating individual neurons as the only explanatory units.

Goodfire’s earlier approach

Goodfire itself applied sparse autoencoders to Llama-3-8B in 2024. Its researchers combined automated feature labels with interventions intended to steer responses, while noting labeling difficulties around special tokens and other influential features. That preview is historical: the company’s page marks its demonstration interface and API as deprecated in February 2026.

What Nanda disputes

In his published response, DeepMind’s Neel Nanda welcomes neuron-based investigations and agent-assisted analysis for practical questions. He rejects the inference that interpretability is nearly solved. He expects important errors and omissions in simplified attribution graphs, argues that language models store more facts than neurons, and believes weak activations may carry distributed information. His central distinction is between useful explanations of selected behavior and confidence that nothing important has been missed.