arXiv cs.LGPaper
Adversarial Self-Distillation
The insight here is structural: template-based safety training doesn't generalize to adversarial input spaces because it's supervised by fixed outputs, not by reasoning. Self-distillation forces the model to learn why it should refuse, not just when, by training on the distribution of prefixes the attacker actually explores. If you're shipping a safety-critical model, this is worth testing against your threat model because it handles a known failure mode that existing defenses don't.