Superposition and Interference

Only $n$ vectors can be perfectly orthogonal in $n$ dimensions. A model that packs more features into the space must reuse directions that overlap. When one feature activates, it creates a small false signal along another direction. That is interference.

ℹ

Interference

For normalized feature directions $f_i$ and $f_j$ , the dot product $f_i \cdot f_j$ measures overlap. Zero means orthogonal. Larger absolute values mean more cross-talk.

Transformer Circuits Thread

Toy Models of Superposition

Trains small ReLU networks on synthetic sparse data to study when and how neural networks represent more features than they have dimensions, and how superposition relates to interference and polysemanticity.

Transformer Circuits ThreadRead the article

Create a Feature Collision

Change the angle between a rare fraud feature and a travel feature. When travel activates, overlap creates a false fraud signal.

Angle between features45°Travel feature active

Direction overlap

0.71

False fraud signal

0.71

Interference is the measurable cost of reusing representational directions.

Interference can explain brittle behavior. A rare fraud feature may overlap with a common travel feature, causing occasional false positives. The model accepts that cost if the extra representational capacity improves average loss.

Checkpoint

Why does sparse feature activation make interference more tolerable?

The superposition hypothesis says models represent more features than they have dimensions by placing features in almost-orthogonal directions. In a toy two-dimensional model, important dense features may get clean axes. As feature sparsity rises, additional directions appear around the circle.

An $n$ -dimensional space can contain only $n$ perfectly orthogonal directions, but high-dimensional spaces can contain many more directions that are nearly orthogonal. If most features are sparse, the model can tolerate occasional interference in exchange for representing many more useful features.

That is why sparse features matter. Most images do not contain dog heads, most images do not contain a curve at one exact orientation, and most tokens do not refer to pizza. Rare features collide less often.

ℹ

Capacity tradeoff

Superposition is worthwhile when the performance gained by representing extra sparse features exceeds the loss caused by interference between their directions.

Pack More Features Than Dimensions

This toy model has two dimensions. Increase feature sparsity and it becomes worthwhile to pack more feature directions into the same space.

Feature sparsity75%

Dimensions

Features kept

Pairwise angle

51°

Interference

0.325

At low sparsity, features often co-occur and need clean axes. At high sparsity, collisions are rare enough that extra capacity wins.

Superposition trades clean separation for capacity. Sparse features rarely activate together, so the model can tolerate some interference.

This hypothesis explains why scaling neurons alone does not guarantee neuron-level interpretability. A larger model may use the extra capacity to represent even more features. Resolving the mixture requires an overcomplete feature basis rather than a longer list of neuron labels.

A dense model tends to keep only the most important features cleanly. As features become sparser, the model starts using superposition: less important features overlap first, then more important ones are pulled into the geometry as the capacity tradeoff changes.

The visual diagnostics include feature-vector norms, cosine similarities, and feature-by-feature matrices. A norm near one suggests a feature is represented; overlap in the matrix shows where features share directions and create interference.

Pack 20 Features Into 5 Slots

Select a feature below, then click a bin to place it. There are only five bins for twenty features — some bins must hold more than one. Darker features matter more to the model; lighter features are less important.

Less importantMore important

Features placed

0 / 20

Bins available

Most crowded bin

A dedicated slot per feature would need 20 bins. With only 5, the model has to let bins carry more than one feature at once — and it protects the darker, more important features first, leaving the lightest features to absorb the most overlap. That overlap is superposition.

Checkpoint

In the lab, why does the model keep more features as sparsity rises?

Superposition experiment — At the top, we have monosemanticity. When our model is dense, all neurons are dedicated to a single feature. As we increase sparsity, polysemantic neurons start to emerge, starting with the features with lowest importance. Eventually, all neurons become highly polysemantic. [Source]

Neuron-based explanations assume the model's coordinate axes line up with meaningful concepts. Superposition breaks that assumption. A feature may use many neurons, and one neuron may support many features.

The axis is convenient for us. Training had no obligation to make it semantically tidy.

ℹ

Privileged basis

A basis is privileged when the architecture or optimization gives features an incentive to align with its coordinate axes. ReLU and sparsity can create some alignment, but they do not guarantee one feature per neuron.

ℹ

How to resolve superposition? 3 hypotheses:

Create models that avoid or reduce superposition.
Find an overcomplete basis that recovers the features inside a model that uses superposition.
Use hybrid approaches that make models easier for a second interpretability stage to analyze.

←PreviousNeurons and PolysemanticityThe Superposition Hypothesis Next→Sparse RepresentationsRepresentation Learning