Giter VIP home page Giter VIP logo

Comments (12)

fra31 avatar fra31 commented on June 18, 2024

Hi,

how many classes does your dataset have? It seems that only 2 logits are available.

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

it is binary. smiling=1 and not smiling=0

from auto-attack.

fra31 avatar fra31 commented on June 18, 2024

The targeted DLR loss requires at least 4 classes (see also #70), then can't be used directly for binary classification problems. We've recently added flags for this with 45f497a, and I'll try to integrate a replacement for such cases soon.

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

Thanks for the fast Response

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

I am using a model with 4 classes. And I updated the AutoAttack. Now, I am getting this warning:

Warning: it seems that more target classes (9) than possible (3) are used in APGD-T! Also, it seems that too many target classes (9) are used in FAB-T (3 possible)!

Then this error:

using standard version including apgd-ce, apgd-t, fab-t, square
initial accuracy: 90.00%
apgd-ce - 1/1 - 445 out of 450 successfully perturbed
robust accuracy after APGD-CE: 1.00% (total time 226.9 s)
Traceback (most recent call last):
  File "/home/user/adversialml/src/src/attacks.py", line 104, in <module>
    adv_complete, max_nr = adversary.run_standard_evaluation(x_test, y_test, bs=args.batch_size)
  File "/home/user/adversialml/src/src/submodules/autoattack/autoattack/autoattack.py", line 172, in run_standard_evaluation
    adv_curr = self.apgd_targeted.perturb(x, y) #cheap=True
  File "/home/user/adversialml/src/src/submodules/autoattack/autoattack/autopgd_base.py", line 679, in perturb
    self.y_target = output.sort(dim=1)[1][:, -target_class]
IndexError: index -5 is out of bounds for dimension 1 with size 4
Load modules...

I think that at least 10 classes are needed and properly tested ?

from auto-attack.

fra31 avatar fra31 commented on June 18, 2024

The number of target classes for the targeted versions of the attacks should be at most equal to the number of classes - 1. For example, for APGD you can specify this with

n_target_classes=9,

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

Ok. When I set n_target_classes=3. Still same problem. Although I trained the model with 4 classes.

from auto-attack.

fra31 avatar fra31 commented on June 18, 2024

Is it possible that it gets overwritten by this?

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

Yes, Thanks for your help. I did changes here:

if version == 'standard':

My changes of n_targeted_classes. I set all to 3 by 4 classes. If this is correct I am not sure, but I don't get a warning anymore.

        if version == 'standard':
            self.attacks_to_run = ['apgd-ce', 'apgd-t', 'fab-t', 'square']
            if self.norm in ['Linf', 'L2']:
                self.apgd.n_restarts = 1
                self.apgd_targeted.n_target_classes = 3 # 9
            elif self.norm in ['L1']:
                self.apgd.use_largereps = True
                self.apgd_targeted.use_largereps = True
                self.apgd.n_restarts = 5
                self.apgd_targeted.n_target_classes = 3 # 5
            self.fab.n_restarts = 1
            self.apgd_targeted.n_restarts = 1
            self.fab.n_target_classes = 3 # 9
            #self.apgd_targeted.n_target_classes = 9
            self.square.n_queries = 5000

from auto-attack.

fra31 avatar fra31 commented on June 18, 2024

Yeah, it should be correct.

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

The corresponding output:

using standard version including apgd-ce, apgd-t, fab-t, square
initial accuracy: 91.60%
apgd-ce - 1/1 - 453 out of 458 successfully perturbed
robust accuracy after APGD-CE: 1.00% (total time 110.8 s)
apgd-t - 1/1 - 2 out of 5 successfully perturbed
robust accuracy after APGD-T: 0.60% (total time 121.9 s)
fab-t - 1/1 - 0 out of 3 successfully perturbed
robust accuracy after FAB-T: 0.60% (total time 140.2 s)
square - 1/1 - 0 out of 3 successfully perturbed
robust accuracy after SQUARE: 0.60% (total time 212.3 s)
max Linf perturbation: 0.01569, nan in tensor: 0, max: 1.00000, min: 0.00000
robust accuracy: 0.60%

from auto-attack.

jS5t3r avatar jS5t3r commented on June 18, 2024

closed.

from auto-attack.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.