Generalized Uncertainty of Deep Neural Networks - Taxonomy and Applications

We review existing and potential methods for improving the efficiency and robustness of learning systems by exploiting the uncertainty of deep neural networks.

SoTeacher: Toward Student-Oriented Teacher Network Training For Knowledge Distillation

How to train an ideal teacher for knowledge distillation? We call attention to the discrepancy between the current teacher training practice and an ideal teacher training objective dedicated to student learning, and study the theoretical and practical feasibility of student-oriented teacher training.

Data Quality Matters For Adversarial Training: An Empirical Study

We show that those well-known problems in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off, are all related to low-quality samples in the dataset. Removing those low-quality samples can greatly alleviate these problems and often boost the robustness as well.