Network Dissection

Network dissection asks whether individual convolutional units align with labeled visual concepts such as trees, stripes, heads, or doors.

It turns a unit's activation map into a binary mask and compares that mask with human-labeled concept masks.

Network dissection diagram — Source: Bau et al. (2017)

ℹ

Intersection over Union

Alignment is measured with:

$IoU=\frac{|M_{unit}\cap M_{concept}|}{|M_{unit}\cup M_{concept}|}$

A high score means the unit activates in many of the same pixels labeled with the concept.

Walk Through Network Dissection

Step through the pipeline that turns one convolutional channel's activations into a concept label — from a forward pass, to a data-driven threshold, to a binary mask, to an IoU comparison against human-labeled concepts.

Ak(x) — raw activation map

Image x is forward propagated through the network to the target convolutional layer.

The pixel activations of channel Channel 37, written Ak(x), are extracted at that layer. Darker blue means a stronger response.

This step repeats for every image in the dataset and every channel — one activation map per channel/image pair.

Bau et al. (2017) apply this per channel, per image, over a large annotated dataset. Here one image and three channels stand in for that process so every quantity stays visible.

The method communicates well because the labels are concrete. It requires expensive pixel-level annotations, many units map to no available concept, and one concept can appear across several units.

Remember that the concept library limits what can be found. “No match” can mean the unit represents something outside the annotation set.

Checkpoint

Why does a low IoU fail to prove that a unit is meaningless?

←PreviousConcept Bottleneck ModelsConcept-Based Explanations Next→Feature VisualizationRepresentation-Based Explanations