Comments (12)
Hi,
how many classes does your dataset have? It seems that only 2 logits are available.
from auto-attack.
it is binary. smiling=1 and not smiling=0
from auto-attack.
The targeted DLR loss requires at least 4 classes (see also #70), then can't be used directly for binary classification problems. We've recently added flags for this with 45f497a, and I'll try to integrate a replacement for such cases soon.
from auto-attack.
Thanks for the fast Response
from auto-attack.
I am using a model with 4 classes. And I updated the AutoAttack. Now, I am getting this warning:
Warning: it seems that more target classes (9) than possible (3) are used in APGD-T! Also, it seems that too many target classes (9) are used in FAB-T (3 possible)!
Then this error:
using standard version including apgd-ce, apgd-t, fab-t, square
initial accuracy: 90.00%
apgd-ce - 1/1 - 445 out of 450 successfully perturbed
robust accuracy after APGD-CE: 1.00% (total time 226.9 s)
Traceback (most recent call last):
File "/home/user/adversialml/src/src/attacks.py", line 104, in <module>
adv_complete, max_nr = adversary.run_standard_evaluation(x_test, y_test, bs=args.batch_size)
File "/home/user/adversialml/src/src/submodules/autoattack/autoattack/autoattack.py", line 172, in run_standard_evaluation
adv_curr = self.apgd_targeted.perturb(x, y) #cheap=True
File "/home/user/adversialml/src/src/submodules/autoattack/autoattack/autopgd_base.py", line 679, in perturb
self.y_target = output.sort(dim=1)[1][:, -target_class]
IndexError: index -5 is out of bounds for dimension 1 with size 4
Load modules...
I think that at least 10 classes are needed and properly tested ?
from auto-attack.
The number of target classes for the targeted versions of the attacks should be at most equal to the number of classes - 1. For example, for APGD you can specify this with
auto-attack/autoattack/autopgd_base.py
Line 589 in 6482e4d
from auto-attack.
Ok. When I set n_target_classes=3
. Still same problem. Although I trained the model with 4 classes.
from auto-attack.
Is it possible that it gets overwritten by this?
from auto-attack.
Yes, Thanks for your help. I did changes here:
auto-attack/autoattack/autoattack.py
Line 265 in 6482e4d
My changes of n_targeted_classes
. I set all to 3 by 4 classes. If this is correct I am not sure, but I don't get a warning anymore.
if version == 'standard':
self.attacks_to_run = ['apgd-ce', 'apgd-t', 'fab-t', 'square']
if self.norm in ['Linf', 'L2']:
self.apgd.n_restarts = 1
self.apgd_targeted.n_target_classes = 3 # 9
elif self.norm in ['L1']:
self.apgd.use_largereps = True
self.apgd_targeted.use_largereps = True
self.apgd.n_restarts = 5
self.apgd_targeted.n_target_classes = 3 # 5
self.fab.n_restarts = 1
self.apgd_targeted.n_restarts = 1
self.fab.n_target_classes = 3 # 9
#self.apgd_targeted.n_target_classes = 9
self.square.n_queries = 5000
from auto-attack.
Yeah, it should be correct.
from auto-attack.
The corresponding output:
using standard version including apgd-ce, apgd-t, fab-t, square
initial accuracy: 91.60%
apgd-ce - 1/1 - 453 out of 458 successfully perturbed
robust accuracy after APGD-CE: 1.00% (total time 110.8 s)
apgd-t - 1/1 - 2 out of 5 successfully perturbed
robust accuracy after APGD-T: 0.60% (total time 121.9 s)
fab-t - 1/1 - 0 out of 3 successfully perturbed
robust accuracy after FAB-T: 0.60% (total time 140.2 s)
square - 1/1 - 0 out of 3 successfully perturbed
robust accuracy after SQUARE: 0.60% (total time 212.3 s)
max Linf perturbation: 0.01569, nan in tensor: 0, max: 1.00000, min: 0.00000
robust accuracy: 0.60%
from auto-attack.
closed.
from auto-attack.
Related Issues (20)
- Can AutoAttack be used to dense prediction task? HOT 2
- Softmax probabilities instead of logits HOT 3
- normalization with deepfool fmodel HOT 4
- About `n_iter`, to align with others HOT 4
- Question about variation in reported (clean & robust accuracy) metrics HOT 2
- Import error for pytorch version 1.10.0 HOT 3
- 4 classes model; targeted attacks are different HOT 2
- A Bug in Square Attack HOT 4
- Problem that forward output is tuple rather than tensor HOT 1
- "The connection to the C10d store has failed" on distributed evaluation HOT 2
- ensemble attack HOT 3
- max Linf perturbation is larger than the epsilon HOT 3
- gradient computation issue HOT 4
- Unable to use Python debugger HOT 1
- Unable to use in TF2 HOT 5
- Parallelized computing HOT 5
- Invalid configuration for square attack HOT 2
- Using the tool for general interval of input region HOT 3
- TF1: How do labels get modified by batch_datapoint_idcs? HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from auto-attack.