Reproduce results about fast_adversarial HOT 1 CLOSED

Hi, sorry for the delay in response here! The default arguments in the code here should correspond to the hyperparameter settings for just achieving the baseline 45% robust accuracy (i.e. Figure 2 in the paper). The only change you should need to make to close the gap is the number of epochs you train for - 30 epochs for FGSM, 40 epochs for PGD, 12 epochs for Free training (see Table 7 in the appendix for these numbers, and divide the Free training epochs in the paper by 8, the number of minibatch replays).

As for your question about Table 3, another speedup from the DAWNBench competition is a smaller architecture size; for our experiments we used a PreAct ResNet18 architecture whereas the PGD-7 in Madry et al. 2017 used a Wide ResNet 18-10, which is much larger.

from fast_adversarial.