sclbd / backdoorbench Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
Great work, but I can't find the link of the paper of Ft_Trojan. Could you please provide it
Hi BackdooorBench Team,
I think I have found a small issue with the backdoor poison index selection method. In the case where a clean label attack is used, the index selection method errors in some cases. I believe the issue is caused by the pratio not being multiplied by the correct length operator. The issue is on line 59 of "backdoor_generate_poison_index.py." I have provided a code snippet below of the issue and how I think it should be fixed. Notice that pratio should be multiplied by len(np.where(targets == tlabel)[0]) rather then len(targets).
non_zero_array = np.random.choice(np.where(targets == tlabel)[0], round(pratio * len(targets)), replace = False)
non_zero_array = np.random.choice(np.where(targets == tlabel)[0], round(pratio * len(np.where(targets == tlabel)[0])), replace = False)
I see this repo has just added a new defense method CLP - Channel Lipschitzness-based Pruning.
Can you update the Leaderboard to include the results of the CLP and README file to update your API?
Thank you for such comprehensive work.
V2.0版本中Label Consistent的数据加载存在一些问题。
在LS攻击的毒化数据生成代码craft_adv_dataset.py中,图像和标签被统一保存了下来,并以npz格式存储,然而在Label Consistent中的89行和95行默认读取的文件格式为npy。
craft_adv_dataset.py中将图像和标签统一保存为npz格式:
BackdoorBench/resource/label-consistent/craft_adv_dataset.py
Lines 775 to 778 in 9df0c6c
Lines 89 to 92 in 9df0c6c
Thank you for your excellent work. I found that the hyperparameters --lr_finetuning_init and --lr_unlearning_init seem to be not used in the ./defense/abl/abl.py. Is that a bug?
Thanks so much for the v2.0 update - it's a big quality of life improvement!
Could you update the readme to include scripts for running an attack and defense with v2.0 (or update saying that the same script as v1 work).
Edit: nevermind! I just got confused by the formatting!
Many thanks for your excellent work! 😁
Now I have a question. When I used the following setting in Blended Attack.
attack_label_trans: all2all
attack_label_shift_amount: 1
num_classses: 10
But I ran into the following error.
Traceback (most recent call last):
File "./attack/laser_attack.py", line 347, in <module>
main()
File "./attack/laser_attack.py", line 212, in main
adv_train_ds = prepro_cls_DatasetBD(
File "./backdoorbench/attack/../utils/bd_dataset.py", line 63, in __init__
assert len(poison_idx) == len(full_dataset_without_transform)
AssertionError
Did I do something wrong? Looking forward to your reply.
Hi,
It seems that a resource of lc_atyack
is missing:
FileNotFoundError: [Errno 2] No such file or directory: '../resource/label-consistent/data/adv_dataset/cifar10_train.npy
Did I forget to download it?
Thanks!
Assuming I have developed some new features on top of the library, how do I merge them into the existing code when BackdoorBench is updated?
Dear authors,
Thank you very much for providing the complete attack and defense baselines, it is a great work.
You mentioned in README.md: We recommend you to download the data and model from model zoo of our website , but we also provide the whole onedrive link at Backdoor Model as an alternative source for your convenience.
However, I tried both links and they didn't work. model zoo has been unresponsive and Onedrive cannot be opened.
Therefore, I would like to ask you to update the latest accessible links.
Looking forward to your reply.
Thanks.
Hello, thank you for the thorough summary you provided.
I encountered some problems. I imported it in torch-vision.model— and conducted color backdoor attack experiments. Therefore, I wanted to evaluate the defense after the attack, but I found that the contents of my weight file differed from those of Git Hub. The format of attack_result.pt is different, so it cannot be defended directly.
For example, the following problem report.
How can I solve the problem? Thanks
Hi BackdoorBench Team,
Firstly, I want to express my admiration for your project.Thank you very much for putting in the effort to design and make this !
But I encountered while attempting to implement the TrojanNN attack method based on your code. It appears that a 'clean_model.pth' is required within the 'resource' directory for the operation to proceed successfully. However, after a thorough review of both the project documentation and the README file, I couldn't find any mention or inclusion of this essential file. I'm concerned that I might have overlooked something or perhaps the file is indeed missing from the repository.
Could you please shed some light on this matter? Any guidance you can offer or provision of the missing file would be greatly appreciated and would significantly aid in my understanding and application of your work.
Thank you in advance for your time and assistance !
On page 16 of the BackdoorBench NeurIPS2022 paper, I noticed that NC is introduced as a method that first detects the backdoored model and then mitigates it through pruning based on the searched trigger. However, for the code in this repo, I find that you perform unlearning instead of pruning. Is that a common operation for the implementation of NC?
I really appreciate your job.
Excuse me. I cannot generate poison data in ssba.py("--attack_train_replace_imgs_path" and "--attack_test_replace_imgs_path" should receive two path for poisoned training data and poisoned testing data, respectively)
We've already installed the packages in requirement.txt file.
We generated the toxic data based on the command in the readme, but the execution reported an error at step 3.
I want to ask how should I solve this problem. Thanks in advance!
Thanks for your great work! I notice that you mentioned a public leaderboard for evaluating backdoor attacks against different defense methods. However, I can't find the results for tiny imagenet. Will you please update the results for tiny imagenet? Much appreciated!
hi there, i notice that attacks like lc and ssba need files in ./reource, could you please upload the resource folder with google drive or baidu disk? thank you!
Thank you for sharing your work with the community and for all that you do. Your project has made a difference in the lives of many people, and for that, I am truly grateful.
However, I have some concerns regarding the accuracy of the results displayed on the leaderboard. Specifically, I noticed that the success rate of the traditional attack method blended against defense methods such as nad/nc appears to be too high. After attempting to replicate the nad defense against blended attacks, I found that the results were not as good as those reported in the original paper but also not as bad as those presented on the leaderboard.
Can you review the accuracy of the results on the leaderboard to ensure they are reflective of the true performance of the defense methods?
Thank you for your attention to this matter.
Hi,
I think in the WaNet Attack All2All attack case, the code to calculate targets_bd is wrong.
Lines 301 to 302 in 408db06
Reference code from WaNet's GitHub.
if opt.attack_mode == "all2all":
targets_bd = torch.remainder(targets[:num_bd] + 1, opt.num_classes)
Hi, I'm trying to use the BackdoorBench folder of backdoored models to benchmark my defense mechanism for publication, but I cannot access the OneDrive. I get a message saying the link has expired and I do not have access.
Hello, thank you for the thorough summary you provided. I encountered a problem while executing a command; when i executing
there might be an infinite loop in the getattr method within bd_dataset_v2.py. I searched online for a solution and found that reducing the number of num_workers could help. I have already adjusted it to 1, but the problem persists. Could you suggest any other possible solutions?
Thanks!
python ./attack/badnet.py --yaml_path ../config/attack/prototype/cifar10.yaml --patch_mask_path ../resource/badnet/trigger_image.png
Thank you for your great work!
I would like to know why the file "cifar10_preactresnet18_bpp_0_1.zip" in OneDrive, which you are offering, contains too much backdoor train data (>num_total_train_datapoisoning_ratio), and what is the role of cross_test_data folder(the amount of cross_test_data equals to num_total_train_datapoisoning_ratio). Which data should I use to train the poisoned model?
there is no fp in your defense folder.
I really appreciate your job.
I have encountered the error when running all attacks on tiny-imagenet.
INFO:root:Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples
then only maximum number of samples will be applied
2022-10-24:21:36:35 [INFO ] [backdoor_generate_pindex.py:35] Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples
then only maximum number of samples will be applied
INFO:root:Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible
2022-10-24:21:36:35 [INFO ] [backdoor_generate_pindex.py:36] Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible
INFO:root:poison num:0.0,real pratio:0.0
2022-10-24:21:36:35 [INFO ] [backdoor_generate_pindex.py:61] poison num:0.0,real pratio:0.0
No poison sample generated !
I want to ask how should I solve this problem. The current pratio is 0.1, Do I need to drease it to 0.01?
Thanks in advance!
I want to attack my own datasets filled with videos using badnets or others, so can you tell me how to run the codes or modify the corresponding codes?
Thank you for your excellent work! I would like to know the target class of Label-Consistent on Tiny ImageNet in Google Drive adv_img folder that you are offering.
Hello, thanks for your excellent work. But I meet a question when I train the Imagenet dataset. It was killed during the pre-process bd dataset creation. I think it may be caused by RAM limitation. Can you help me to solve it?
Hi BackdoorBench Team,
Firstly, this tool is amazing. Thank you very much for putting in the effort to design and make this.
I have found an issue with the implementation of the ANP method. In both the evaluate_by_threshold and evaluate_by_number methods, the test ASR value and ACC accuracy difference are used to determine what threshold value is used. However, the test ASR value would not be accessible to the defender when performing pruning as the paper does not assume that backdoor data is available. Could you please clarify if this is an issue or if I am missing something.
Thanks :)
when i run the lira attack experimental with the command python ./attack/lira.py --save_folder_name lira
,i got the result like this
i don't know why the test_acc will be so low,and i run it with default pattern, can you tell some Possible causes about it. Thanks!
and another question about the code lira .py
is that i'm not clear understanding the Autoencoder's work,Does it act as a noise canceller?because i found that it doesn't have a loss and backward function
FileNotFoundError: [Errno 2] No such file or directory: '../resource/ssba/cifar10_ssba_train_b1.npy'
Very nice work. May I confirm that the LC attack is Label-Consistent Backdoor Attacks as shown in this project. In the paper, the LC attack cites the paper Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks.
Hi!
There may lack a definition of the get_features
, i.e.,
features, labels = get_features(args, model_attack, target_layer, data_loader)
NameError: name 'get_features' is not defined
Could you provide some suggestions on this issue?
Thanks a lot.
Hi, SCLBD! Thank you for your excellent work! I have a question for the low-frequency attack: How to generate the trigger by myself for the model of a new architecture?
Thank you for sharing your work! It's really helpful.
However (excuse me), I can't generate trigger for low frequency attack with resources/lowFrequency/generate_pattern.py.
There's no errors in execution, but the fooling rate in Universal Perturbation algorithm keeps 0 in 50 epochs (it just doesn't change), which means the perturbation is not optimized and the agorithm is not working.
The picture below shows the fooling rate in epoch 12.
Here's some information:
BackdoorBench/resource/lowFrequency/generate_pattern.py
Lines 268 to 272 in 04137b7
Your work is the only pytorch-based implementation for lf attack that I can find. The triggers you provided is useful and I'm grateful, but I still want to generate my own triggers.
I want to ask how can I solve this problem. Thanks in advance!
First, think about this benchmark, which helps me to reproduce baseline experiment results quickly. However, I have a few concerns about the code in the backdoor_generate_pindex.py. Specifically, we can see that in https://github.com/SCLBD/BackdoorBench/blob/main/utils/backdoor_generate_pindex.py#L59,
non_zero_array = np.random.choice(np.where(targets == tlabel)[0], round(pratio * len(targets)), replace = False)
Seems like the non_zero_array is selected from the label, maybe there should use targets!=tlabels.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.