Feature Discovery
After training a sparse autoencoder, feature discovery begins. Run many inputs through the model, record which learned features activate, and inspect the highest-activating examples. Automated systems can propose labels, but examples and interventions still decide whether the label holds.
Discovery pipeline
Sample activations → train a sparse dictionary → collect top examples → propose a label → test negatives → intervene on the feature.
Feature discovery can uncover safety-relevant concepts, memorized entities, code patterns, and dataset artifacts. It also produces dead features that never fire and split features that divide one concept across several directions. Count and label quality both matter.
What is a dead feature, and why does it matter?
A learned feature is useful when it is interpretable, specific enough to distinguish relevant cases, broad enough to cover the concept, and causally connected to behavior. These goals can conflict.
Evaluation dimensions
- Interpretability: can a person describe it?
- Precision: how many activations match the label?
- Recall: how many relevant examples activate it?
- Causal effect: does intervention change predicted behavior?
Does the Feature Label Hold Up?
Pick a learned feature and set an activation threshold. Compare the proposed label with real activating examples, near misses, and counterexamples.
Precision
1.00
Recall
1.00
For a monitoring feature, choose the threshold from operational costs. A low threshold catches more risky outputs and creates more false alarms. A high threshold reduces review load and may miss subtle failures. Report the tradeoff instead of presenting one magic cutoff.
Why is a high automated interpretability score insufficient for deployment?
A sparse feature library is a catalog of learned directions, activating examples, labels, scores, and intervention results. It turns one-off analysis into shared infrastructure.
The library needs versioning. A feature index from one checkpoint or autoencoder may mean something different after retraining.
Feature record
A useful record includes model and layer, autoencoder version, decoder direction, activation statistics, top and negative examples, proposed labels, evaluation scores, and known causal effects.
A safety team can use the library to search for refusal, deception, or sensitive-entity features. A product team can reuse style and domain features for controls. Governance matters because a persuasive label can outlive the evidence that originally supported it.
What metadata would you require before another team uses a feature as a production monitor?