julian-wyatt / anoddpm Goto Github PK

Hi, Julian.
Sorry to disturb you. The research is so great! I am trying to refactor that on my custom dataset. But I am trapped with the problem about saving .mp4 files during training process. I can not save the .mp4 file by "ani.save(f'{ROOT_DIR}diffusion-videos/ARGS={args["arg_num"]}/sample-EPOCH={epoch}.mp4')"

The error message indicates issues with the ffmpeg encoder, mentioning "incorrect library version loaded' and 'Error initializing output stream 0:0'. It suggests that there might be incorrect parameters such as bit_rate, rate, width, or height."

I have already checked the matplotlib's version is 3.1.1, and also installed some other versions. Does anyone have some idea about this problem? Thanks.

Thank you for your good work(AnoDDPM)!When i try to recurrent your job ,i got some problems can't be solved . It's my honor if you can help me out.
1.In the paper AnoDDPM,figure1,3,5 shows you have ever used gaussian noise for reconstruct in brain tumor datasets and mvtec leather classes.However,when i try to apply your code to have the same result of figure 1,3,5,i found the denoise process is too awful to recognize the original image,not to mention reconstruction.So i'm wondering how you train the DDPM with gaussian.It's better if you can show me the test args you used.Here it is my result

2.In your paper AnoDDPM 3 section methodology part,you explained the reason why gaussian noise can't work.Not intended to be offensive,i can't really understand it ,especially the illustration that"This limits the discriminatory power of an AnoDDPM model as low frequency components are inferred to be relatively corruption free, resulting in large anomalous regions being reconstructed in the reverse process.'

First thank you for your excellent work. I have the question that how long did it take to train the model. I used the same GPU (Titan XP) as mentioned in the paper, and it took me more than one day to train the model on the single subset "leather" on the MVTec dataset.
Can I do changes to accelerate my training?

1. I try to change the batch size since I find there's still a lot of memory room on GPU but facing some code error. It seems something in simplex noise code only support batchsize=1
2. I want to reduce the epoch which is now 3000 to 500 to reduce the training time, but I have no idea whether it will do harm to the final performance

Dear Julian, thank you very much for open-sourcing such a creative code, which has benefited me a lot! After studying your code carefully, I have some confusion about the training of the model. If you have time, can you help me?

1. I want to know how many epochs did you train on the leather dataset? After I trained 3000 epochs, I added noise with a step size of 150, but the model could not reconstruct the original input (on the training set).
2. I want to know if my logic is correct: we set the step size T to 1000 during training, but it is sufficient to set the step size to 150 or 250 during reconstruction and sampling. Is this the case?

I would be very grateful if you could take time out of your busy schedule to answer me. Wish you all the best!

In dataset.py, a directory raw_cleaned is used. I would like to know how to ready the directory.
I understand that I can ready a directory raw by readying ./DATASETS/CancerousDataset/EdinburghDataset and running dataset.py, but this make a raw_new directory. What is the difference between raw_new and raw_cleaned?

Hello dear author! Thank you for your contribution. When I trained model on the leather object, I checked 'htop' command on Linux and found that CPU% became very high. Is this phenomenon normal, or any problem with the code related to memory utilization? I'm Looking forward to your reply. Thank you so much!

In the file "detection.py," I found "import Comparative_models.CE as CE" on line 466. How I can get this package? Moreover, I want to run your model based on the dataset "MVTec". Would you please advise me on how I can reproduce the result? What the configuration file should look like and the steps to reproduce the result.

Hi，this is a nice work！ I have read your paper and is interested in applying Diffusion model to anomaly detection task. Could you add more detailed information about how to train and evaluate AnoDDPM in README file?

Dear author, I found the following characters missing when running the training code"from perlin_numpy import generate_fractal_noise_2d",Could you tell me where I can find this?thank you very much！

Hello,

Would it be possible to get access to the args.json file that was used to train the model in the paper. I know the hyperparameters are in the paper but I am not exactly sure which parameters in the json value correspond to the different betas, EMA etc.

Thanks

Hello, Professor, your work is very wonderful, but I have some questions when I reproduce your paper, that is, why can the batchsize only be 1? When I change it to a different multiple, the model will report an error during the diffusion process

Originally posted by @ZZZGGGG in #16 (comment)

I was wondering why the simplex noise depends on the diffusion timestep t in rand_3d_fixed_T_octaves().

Hello,

I am trying to run this repo with the same data as presented in the paper in order to gain a baseline understanding of this diffusion model. However, I am having trouble setting up the datasets. For the datasets:

1. Are these the links for the training and test sets?

• a. Training to be placed in ./DATASETS/Train directory: http://preprocessed-connectomes-project.org/NFB_skullstripped/
• b. Testing to be placed in ./DATASETS/CancerousDataset/EdinburghDataset/Anomalous-T1/raw directory : https://reshare.ukdataservice.ac.uk/851861/

2. There are a lot of .tar files within the Testing dataset (if that link is correct). Did you decompress the files, open the individual datasets and save them as a .npy file?

3. Will the files have to be read and saved to .npy files within these directories?

4. Do you have a detailed folder structure and file format we need to use in order to get your code working?

Hi! I can't get the test data from the website，can you help me ?

Excuse me, Sir. Thanks for your code that made a great contribution in the computer vision company.I am a master student in China.Could you give me the address of the data.I will appreciate you if you could replay me

Could you please provide the evaluation code for MVTEC dataset or other image dataset to reproduce the AUC results in your paper?
Thanks a lot!!

Issue
When I increase the batch size found in the args28.json file the training errors out.

Expected
I would like to increase the batch size in order to speed up the training process. The current memory usage is 4GB on a total of 16GB VRAM. So I should be able to increase the batch size to 3 or 4.

Reproduce
change the dataset within args28.json to "cifar" and change the Batch_Size to 4

Error Message

Hi!

I was looking for how to use detection.py, In that we have to put the dataset in ./DATASETS/CancerousDataset/EdinburghDataset/Anomalous-T1/raw but in raw folder for all 22 scans which file we have to keep, there are many files present in the one scan. Should it be binary file in Axial_T2 folder for each scan?

Some guidance is helpful!

Thanks!

Why does the noise addition and denoising process not destroy the normal part, but “assimilate” the abnormal part into the normal part. Because the model remembers the features of the normal part while training? Then the reasoning process destroys and transforms the abnormal part？

Hello author, I am honored to read your paper. After completing leather data set training, I tried to detect, but encountered some problems:
Here is my args file:

args19.json
{
"img_size": [256,256],
"Batch_Size": 1,
"EPOCHS": 1000,
"T": 1000,
"base_channels": 128,
"beta_schedule": "linear",
"channel_mults": "",
"loss-type": "l2",
"loss_weight": "none",
"train_start": true,
"lr": 1e-4,
"random_slice": true,
"sample_distance": 600,
"weight_decay": 0.0,
"save_imgs":true,
"save_vids":true,
"dropout":0,
"attention_resolutions":"16,8",
"num_heads":2,
"num_head_channels":-1,
"noise_fn":"simplex",
"dataset":"leather",
"channels": 1
}

After the training:

1. Error is reported in evaluation.testing() immediately after train:

File "diffusion_training.py", line 153, in train
evaluation.testing(testing_dataset_loader, diffusion, ema=ema, args=args, model=model)
File "/home/dell/local/HK/AnoDDPM-master/evaluation.py", line 124, in testing
x = x.to(device)
NameError: name 'device' is not defined

3. When I tried detection, I entered the following command:
   python3 detection.py args19.json
   Here, args19.json is the configuration file I trained with
   But the following error is reported:

This is the shape of the layer for the row where the error occurred
Conv2d(1, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1）

Traceback (most recent call last):
File "detection.py", line 959, in
anomalous_metric_calculation()
File "detection.py", line 226, in anomalous_metric_calculation
t_distance=200, denoise_fn=args["noise_fn"]
File "/home/dell/local/HK/AnoDDPM-master/GaussianDiffusion.py", line 354, in forward_backward
out = self.sample_p(model, x, t_batch, denoise_fn)
File "/home/dell/local/HK/AnoDDPM-master/GaussianDiffusion.py", line 299, in sample_p
out = self.p_mean_variance(model, x_t, t)
File "/home/dell/local/HK/AnoDDPM-master/GaussianDiffusion.py", line 279, in p_mean_variance
estimate_noise = model(x_t, t)
File "/home/dell/anaconda3/envs/pytorch181/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/local/HK/AnoDDPM-master/UNet.py", line 400, in forward
h = module(h, time_embed)
File "/home/dell/anaconda3/envs/pytorch181/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/local/HK/AnoDDPM-master/UNet.py", line 35, in forward
x = layer(x)
File "/home/dell/anaconda3/envs/pytorch181/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/pytorch181/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 399, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/home/dell/anaconda3/envs/pytorch181/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 396, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [128, 1, 3, 3], expected input[1, 3, 256, 256] to have 1 channels, but got 3 channels instead

I wonder if there is something wrong with my detection call (I did not change your code) :

Maybe my question is relatively simple, but it has really caused me a lot of trouble. I would like to express my deep thanks again for troubling you.

thanks for your nice code, I am a beginner in the field of diffusion and hope to learn more through your code，I have some questions during my learning process and hope you can help me.
1、When running your open-source code, I set the 'vis_img' switch to true, expecting to see the images of xt-1 and x0，However, I have some doubts about the images displayed. Why is the image of Xt-1/sample clearer than that of X0/pred_x0? Moreover, I noticed that when using 'vis_img', the noise was set to torch.rand_like, while for 'vis_video', the noise was sampled by noise_fn. Would this cause a difference in the images? Here are my results

I hope you can help me, thanks!!!

Well job! I notice that you use the class 'leather' in MVTec dataset to show the segmentation performance. So, I wonder whether you do a complete experiment on MVTec and report the image/pixel AUROC?

In the file "detection.py," I found "import Comparative_models.CE as CE" on line 466 and 684. How I can get this package? I have looked at other people's questions, but I still don't know how to solve them. I noticed that you mentioned deleting its usage, but it was useful in other parts. How should I solve it? Thank you for your answer.

    Please, how can I get the requirements.txt. file？

Originally posted by @kbqn20 in #5 (comment)

Dear Julian,

Thanks for sharing the code. Sorry if I missed it, but I haven't found any information about which packages (incl. versions) your code is relying on. Could you please share a list of install-requirements? That would be great, thanks!

Best,
Philipp

Do you have pre-trained data for medical images?

I understand NFBS repository has 125 sample(e.g., A00028185), and it is split into 100 training data and 25 testing data.
I want to know names of 25 testing data(e.g., A00028185) for more detailed reproduction.

Thank you for your work (AnoDDPM), the work is very great. I have some confusion about the work.​

1. The epoch set in your parameters is 3000. How long did it take you to run these 3000 epochs with a single NVIDIA Titan Xp GPU with 12GB GDDR5 in your environment.
2. A neuroimaging dataset of brain tube patients (your test datasets) cannot be downloaded. Is it convenient for you to share the dataset?

Looking forward to your reply.

Hello, you're doing a great job. However, I've encountered an issue with the line "import Comparative_models.CE as CE". I've looked at questions left by other people, but it might be due to my own problem that I still don't know how to obtain this module. I would greatly appreciate your guidance if you have the time. Thank you very much.😊

Dear Julian,

I hope this message finds you well. I am a student specializing in anomaly detection at Lanzhou University of Technology.
I attempted to contact you through your email address [email protected], but unfortunately, I encountered access issues. In order to ensure that I can get in touch with you, I am reaching out via this method. I appreciate your understanding.

If it's convenient for you, please feel free to reply to this message, or provide an alternative contact method where I can reach you.
I have been following your work on the AnoDDPm model with great interest. I kindly request your assistance in obtaining the checkpoints of the AnoDDPm model. Access to these checkpoints would significantly benefit my research and help me better understand its implementation.

I deeply appreciate your contributions to the field and would be grateful for your support. Rest assured, I will use the resources responsibly and provide proper attribution.

Thank you for your time and consideration.

Sincerely,
Zhiqiang Feng
Lanzhou University of Technology
[email protected]

Hi I tried to run the introduced model using MVtec dataset according to READ.ME, but I got the following error in detection.py. I don't know how to solve this problem. Please help me to solve this problem.

The test_arg I used is as follows

{ "img_size": [ 256, 256 ], "Batch_Size": 1, "EPOCHS": 3000, "T": 1000, "base_channels": 128, "beta_schedule": "linear", "channel_mults": "", "loss-type": "l2", "loss_weight": "none", "train_start": true, "lr": 1e-4, "random_slice": true, "sample_distance": 600, "weight_decay": 0.0, "save_imgs": false, "save_vids": true, "dropout": 0, "attention_resolutions": "16,8", "num_heads": 2, "num_head_channels": -1, "noise_fn": "gauss", "dataset": "leather", "channels": 3 }

The error statement is as follows.
RuntimeError: Error(s) in loading state_dict for UNetModel: size mismatch for down.0.0.weight: copying a param with shape torch.Size([128, 3, 3, 3]) from checkpoint, the shape in current model is torch. size mismatch for out.2.weight: copying a param with shape torch.Size([3, 128, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 128, 3, 3]). size mismatch for out.2.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([1]).

Waiting for a solution.

Hello author, when I run the detection code with in_channel==3, I encounter the following problem:
Traceback (most recent call last):
File "detection.py", line 958, in
anomalous_metric_calculation()
File "detection.py", line 231, in anomalous_metric_calculation
fpr_simplex, tpr_simplex, _ = evaluation.ROC_AUC(mask.to(torch.uint8), mse)
File "/home/primary/4T_a/AnoDDPM-master/evaluation.py", line 81, in ROC_AUC
return roc_curve(real_mask.detach().cpu().numpy().flatten(), square_error.detach().cpu().numpy().flatten())
File "/home/primary/anaconda3/envs/zg/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 963, in roc_curve
y_true, y_score, pos_label=pos_label, sample_weight=sample_weight
File "/home/primary/anaconda3/envs/zg/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 733, in _binary_clf_curve
check_consistent_length(y_true, y_score, sample_weight)
File "/home/primary/anaconda3/envs/zg/lib/python3.7/site-packages/sklearn/utils/validation.py", line 334, in check_consistent_length
% [int(l) for l in lengths]
ValueError: Found input variables with inconsistent numbers of samples: [65536, 196608]

My mask is (1,1,256,256,), but my mse is (1,3,256,256), and I get an error when calculating roc_auc, my argument is:
{
"img_size": [
256,
256
],
"Batch_Size": 1,
"EPOCHS": 3000,
"T": 1000,
"base_channels": 128,
"beta_schedule": "linear",
"channel_mults": "",
"loss-type": "l2",
"loss_weight": "none",
"train_start": true,
"lr": 1e-4,
"random_slice": true,
"sample_distance": 800,
"weight_decay": 0.0,
"save_imgs": false,
"save_vids": true,
"dropout": 0,
"attention_resolutions": "16,8",
"num_heads": 2,
"num_head_channels": -1,
"noise_fn": "simplex",
"dataset": "leather",
"channels":3
}
Is there a problem with where I did it？

Hello.
I want to reproduce your work.
Please let me know args file to train the model using simplex noise in Table 1 in your paper.
Is it test_args/args28.json ?
My apologies if this has already been mentioned somewhere.