Attention vs. Attribution
Attention asks where a component reads. Attribution asks how input features relate to the output. Those questions can point to different tokens.
For a sentiment prediction, a head may attend to sentence structure while integrated gradients assigns most output relevance to “excellent.”
Two explanatory targets
Attention: normalized internal mixing weights between positions. Attribution: an estimate of how inputs contribute to or affect a selected output. Agreement is informative; disagreement is also informative.
Compare Routing with Output Evidence
Attention and integrated-gradient attribution disagree on this positive review. Remove a disputed token to test which signal better predicts output change.
Positive sentiment
0.93
Largest measured drop
run a test
Do not pick the method with the prettier heatmap. Decide whether you need to inspect routing, identify sensitive inputs, or test a causal mechanism.
When methods disagree, perturb the disputed tokens and inspect the relevant internal pathway.
What should you do when attention highlights “nurse” but attribution highlights “won”?