p2333 / bag-of-tricks-for-at Goto Github PK

In the original Trades, it uses the beta to trade the adv loss and clean loss. But I did not find beta in the code?

Hi! I tested the provided checkpoints on both WRN-34-20 and WRN-34-10 and did not get the same results as the paper. Both are around 60+%. Are they the right checkpoints? Thanks!

Was trying to reproduce your reported PGD-10 results. if I removed the normalize function when evaluating PGD, I got 0.8566 which is close to the clean accuracy, but I expected it to be 50+. If I did not remove the normalize function, I got 0.2319, which was also unexpected. Can you take a look at the provided checkpoints? thanks!

Also, just want to be sure, the PGD-10 you guys performed during evaluation in the paper is the default PGD-10? Meaning, attack_iter = 10, restarts=1, eps=8, step=2? Essentially, I just have a problem reproducing some of the results. Thank you!

How much epoch is used in Trades in this paper ? And if the learning rate is the same as the original Trades?

Hello, Pang.

Thanks for your sharing.

Congratulations on your contributions to robustness research.

I tried to run you empirical engineering trick progrm in default setting and run the evalution program from you AT_With_He paper.
https://github.com/AnonymousForDoubleBlind/AT_with_HE/blob/master/CIFAR-10/pgd_attack_cifar10.py

However, it is very wierd that the PGD err is 0.85 while natural err is 0.81.

If I wnt to use the pgd_attack_cifar10.py, what other step should I do to fix the bug so that correct results can be demonstrated.

Thank & Regards!
Momo

Have you tried adversarial training without dataset normalization, which means Input <- (Input - mean) / std?

I tried TRADES on CIFAR10 without dataset normalization, and couldn't get a comparable result.

So I wonder if dataset normalization is the problem.

https://github.com/P2333/Bag-of-Tricks-for-AT/blob/master/train_cifar.py#L186

Does the model's gradient accumulats when adversarial examples is computed?