Defensive Distillation
Defensive distillation is a technique proposed by Papernot et al. (2016) to improve the robustness of machine learning models against adversarial examples.
The goal is to transfer knowledge from a highly regularized, robust teacher model to a more compact student model, while preserving the teacher's robustness properties.
Process:
- Train a teacher model using adversarial training or other robust training techniques.
- Use the teacher model to label a clean dataset, producing soft labels (probability vectors) instead of hard class labels.
- Train a student model on the same clean dataset, but using the soft labels from the teacher as targets.
- The student model learns to mimic the teacher's probability distributions, inheriting its robustness to adversarial examples.
Why Soft Labels Matter
A hard label says only which class an input belongs to. A soft label, the teacher's full probability vector, also encodes how close the input is to other classes. Training the student on that richer target smooths the decision surface it learns, which is the property distillation is trying to transfer.
A Broken Defense
The original formulation of defensive distillation was shown to be largely ineffective against stronger attacks, particularly the Carlini & Wagner (C&W) attack. The technique mainly succeeded at masking gradients rather than removing adversarial vulnerability, which gave a false sense of security against gradient-based attacks while leaving the model exploitable by attacks designed around that masking.
The lesson generalizes beyond this one technique: a defense that only reduces the effectiveness of the attacks used to evaluate it has not been shown robust. Evaluation must include adaptive attacks, ones specifically designed with knowledge of the defense, before a robustness claim can be trusted.
In defensive distillation, why does the student model train on the teacher's soft labels instead of the original hard class labels?