Double Descent in Adversarial Training: An Implicit Label Noise Perspective
Here, we show that the robust overfitting shall be viewed as the early part of an epoch-wise double descent -- the robust test error will start to decrease again after training the model for a considerable number of epochs. Inspired by our …
Data Quality Matters For Adversarial Training: An Empirical Study
Multiple problems in adversarial training including robustness-accuracy trade-off, robust overfitting, and gradient masking share one commnon cause -- low quality samples in the dataset.