Evidence

A convincing circuit has several kinds of evidence. Feature visualization suggests what components detect. Weight inspection shows how they connect. Dataset examples show when they activate. Ablation or patching tests whether the behavior depends on them.

ℹ

Converging evidence

Strong mechanistic evidence combines observational, structural, and causal results. Each method covers a weakness in the others.

MI-specific evidence: feature visualization, implementation, dataset examples, feature use, and handwritten circuits — MI-specific evidence stack

Build an Evidence Stack

Turn on the evidence available for a proposed pose-invariant dog-head detector. The conclusion should become more specific as independent methods agree.

Evidence types

1 / 4

Claim strength

suggestive

Converging evidence is stronger because each method fails in a different way.

Distill

Zoom In: An Introduction to Circuits

An introduction to the circuits approach to mechanistic interpretability, arguing that neural networks can be understood through the features they build and the circuits that connect them.

DistillRead the article

For a curve circuit, researchers can inspect upstream oriented edges, read signed weights, find downstream circle detectors, and hand-set weights that reimplement the behavior. That chain is much stronger than naming a neuron from its top image.

Evidence for curve detector features and circuits — Evidence for curve detector circuits: earlier curves and lines combine to form curves, which downstream become more complex curves and shapes. [Source].

Pose-invariant dog-head detectors are a second example of evidence for circuits. Feature visualization can suggest the sensitivity of a neuron, while dataset examples test how that sensitivity appears in natural images and whether another stimulus also activates it.

Pose-invariant dog-head detector evidence — Pose-invariant dog-head detector. [Source].

Cars in superposition introduce a limit case for circuit evidence. The circuit can still compute useful car-related features, but a single neuron may not carry one clean semantic label. That example points forward to the superposition hypothesis.

Car circuit — Circuit for a car detector. [Source].

Showing cars in superposition — Cars in superposition (we will learn more in the upcoming chapter on the superposition hypothesis). [Source].

Checkpoint

Which result would most weaken a proposed circuit: low activation on one example, or no behavioral effect after a targeted intervention?

A circuit-based explanation should read like an executable account: this feature is detected, that head routes it, this downstream feature combines it with another signal, and the output logit rises.

ℹ

Useful explanation

A useful circuit explanation is faithful enough to predict behavior, compact enough for a person to inspect, and scoped to the inputs and task where it was validated.

Predict an Invoice Circuit

Adjust three document features and use the visible weights to predict the invoice score before reading the output.

Invoice-like layout0.80Currency amounts0.70Signature block0.40

score = .45(0.80) + .40(0.70) + .25(0.40) - .15

Invoice score

0.590

Classification

invoice

A circuit explanation earns trust when it lets you predict how a controlled input change will propagate.

Checkpoint

What scope statement should accompany a circuit found on short English prompts?

←PreviousMechanismsCircuits Next→Neurons and PolysemanticityThe Superposition Hypothesis