juliachang / sc-cam Goto Github PK

Hello, we met the "core dumped" problem during clustering. Which version of scikit-learn shoud be used?

I want to modify the images used for training: i want to add my own images. how do i go around doing that?
here is what i did:

1. I added the raw images to the JPEGImages folder
2. I added the segmented versions to the SegmentationClassAug folder
3. I added the image paths to train_aug.txt in voc12
4. I added their labels to train_label.npy but not sure I added them correctly
5. I added their image name and class label to cls_labels.npy in the same format as the other entires
6. did not need to add them to 20_class_labels.npy, as they were already there
   I added about 1500 images, extract features extracts all the 12000+ images, but when i run create pseudo label, in the line instead of printing (12000, 200) (12000, 20) it prints (18000, 200) (18000, 20)
   What am i doing wrong, or am i missing any steps?

Heres the error:

18605 18605
18605 18605 (16458, 20)
Traceback (most recent call last):
  File "create_pseudo_label.py", line 202, in <module>
    train_filename_list, train_label_200, train_label_20 = create_train_data(merge_filename_list, new_label_list, keep_idx_list)
  File "create_pseudo_label.py", line 126, in create_train_data
    train_label_20.append(label_20[idx])
IndexError: index 16460 is out of bounds for axis 0 with size 16458
k_cluster: 10

Is the author's classification based on siamese network?

Hi. I got all black image when I run infer_aff.py file. Does anyone have the same problem as me？please help me，thank you!

Where is the source code of ResNet-38 in the project？
is it the source code in this project?
The link to "ademxapp" , i didn't find the code about network, it just the model file.
Where can i find the source code of ResNet-38 network of pytorch?
Thx.

Excuse me. Sorry to bother you, you may be busy with building this project and others. Although the project has not been released, I have a question about it. Did you use the official deeplabv2 resnet-101 model and load COCO pre-train weight during training? Can you share your training method of your deeplabv2 resnet-101 with us? It's fine to reply to me when you are free. If you think this is not an issue, I can close it.

DO you change any parameters or model architecture using this repro?

class 0 background IU 0.76
class 1 aeroplane IU 0.39
class 2 bicycle IU 0.30
class 3 bird IU 0.40
class 4 boat IU 0.36
class 5 bottle IU 0.55
class 6 bus IU 0.69
class 7 car IU 0.54
class 8 cat IU 0.62
class 9 chair IU 0.28
class 10 cow IU 0.51
class 11 diningtable IU 0.46
class 12 dog IU 0.57
class 13 horse IU 0.59
class 14 motorbike IU 0.59
class 15 person IU 0.49
class 16 pottedplant IU 0.43
class 17 sheep IU 0.56
class 18 sofa IU 0.50
class 19 train IU 0.53
class 20 tvmonitor IU 0.47
meanIOU: 0.5028162179859049

class 0 background IU 0.62
class 1 aeroplane IU 0.23
class 2 bicycle IU 0.23
class 3 bird IU 0.21
class 4 boat IU 0.30
class 5 bottle IU 0.51
class 6 bus IU 0.55
class 7 car IU 0.39
class 8 cat IU 0.36
class 9 chair IU 0.20
class 10 cow IU 0.41
class 11 diningtable IU 0.34
class 12 dog IU 0.28
class 13 horse IU 0.39
class 14 motorbike IU 0.41
class 15 person IU 0.26
class 16 pottedplant IU 0.38
class 17 sheep IU 0.36
class 18 sofa IU 0.42
class 19 train IU 0.47
class 20 tvmonitor IU 0.39
meanIOU: 0.36679235052055154

The model outputs x_20 and y_20
What are these exactly?
same for x_200 and y_200

Also x_200 / y_200 has a size of 200 per image. can i assume, the first 10 values are for the first class, the next 10 values for the second class and so on?

Also i set the value for k_cluster in script.sh to 5, but only label_200 came out as size 100 for each image. x_200 and y_200 were still size 200 for each image. Why?

Hi
I am getting a lot of nan arrays or arrays with very large numbers when extracting features from round 1
Any idea what is causing this?

Thank you for sharing the code.

As I understand, the models for "extracting feature step" and "classification step" are resnet-38, which has same architecture except additional fully connected layer at the last part.

However, the initial weight for "extracting feature step" and "classification step" are different. One is from Resnet38(AffinityNet), another is from Resnet38(MXNET).

Is there special reason for different initial weights?

Or how about using same initial weights?

Hi, thanks a lot for sharing your very interesting work.

I am running the code to try to reproduce the results in the paper (still running the second round). If I understood correctly, the results that I should get after this are the ones shown in the Table (~50.9), which corresponds to the model without the RW in AffinityNet. I was wondering how I should proceed to run this model (SC-CAM + RW) to get the 63.4 mIOU reported in the paper.

Thanks!

Pierre

hi, thanks for your brilliant work firstly.
I met some trouble when re-implement your reported result with the deeplab-v2. May i ask is that the caffe model pretrained on COCO dataset did you used as the initial weight to train the deeplabv2?

Thank you for your great works
About the deeplabv2 repo you provide, how did you initialize the parameters of deeplabv2. Did you just train it from scratch or using the pretrained COCO or VOCAug model. And could you detail your hyperparameters?

For the CAM visualization, I want to use a blue background and red for attention, how do I convert it in the code?

In your paper you mention both ResNet-38 and ResNet-101
Which one is used where?

Could you explain why the code need flip operation in the lines 64 through 66? Several papers hava also used this operation.

if i % 2 == 1: cam = np.flip(cam, axis=-1) return cam

Hello,

Thanks for sharing the great work.

As I using my custom dataset during R0's feature extraction, I had the problem of not having enough GPU ram. I am using two RTX 3090 with 24GB ram each, but I noticed that the work was experimented with a 12GB GPU.

I monitored the vram usage, and noticed that the used vram went up as the for loop going through all the images to extract feature.

I wonder if you had any insight regarding this? Thank you.

thanks for your codes. According to the paper, your method can also improve the performance of classification, but there are no experiments in your paper. Can you provide some experimental results of classification performance?

By iteratively updating the feature extractor, two classifiers, and sub-category pseudo labels, the enhanced features representations lead to better classification

Hi
In your paper, you guys have 2 different mIOU results. One is 49.6, and the other is 65.9
What is the difference between the two?

Great paper! will the code be available? thanks!

Hi
Do you only use the Pascal Voc 2012 dataset with 2900 images
What about the augmented data set that has over 10000+ images here: http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz

Thank you for your contribution, I am currently reproducing your code.

I just modified the --infer_list of the infer_cls.py

python infer_cls.py --infer_list voc12/val.txt --voc12_root ${dataset_folder} --weights ${final_model} --save_path ${save_cam_folder} --save_out_cam 1 --k_cluster 10 --round_nb 3

and get :

class 0 background IU 0.76
class 1 aeroplane IU 0.40
class 2 bicycle IU 0.25
class 3 bird IU 0.38
class 4 boat IU 0.36
class 5 bottle IU 0.49
class 6 bus IU 0.66
class 7 car IU 0.58
class 8 cat IU 0.59
class 9 chair IU 0.27
class 10 cow IU 0.50
class 11 diningtable IU 0.40
class 12 dog IU 0.57
class 13 horse IU 0.54
class 14 motorbike IU 0.59
class 15 person IU 0.51
class 16 pottedplant IU 0.40
class 17 sheep IU 0.55
class 18 sofa IU 0.47
class 19 train IU 0.53
class 20 tvmonitor IU 0.48
meanIOU: 0.48929806252105235

any error here？ In the train.txt, achived 50.9

Does anyone encounter this kind of situation? The final mIoU result is 0 except for the background?

Thanks for a great job! I'm following your job.
When I run your code, I got very strange results. As follows,

class 0 background IU 0.81
class 1 aeroplane IU 0.00
class 2 bicycle IU 0.00
class 3 bird IU 0.00
class 4 boat IU 0.00
class 5 bottle IU 0.00
class 6 bus IU 0.00
class 7 car IU 0.00
class 8 cat IU 0.00
class 9 chair IU 0.00
class 10 cow IU 0.00
class 11 diningtable IU 0.00
class 12 dog IU 0.00
class 13 horse IU 0.00
class 14 motorbike IU 0.00
class 15 person IU 0.00
class 16 pottedplant IU 0.00
class 17 sheep IU 0.00
class 18 sofa IU 0.00
class 19 train IU 0.00
class 20 tvmonitor IU 0.00
meanIOU: 0.2713713433526475

I don't change the network, and I just change some positions of the folder. I would like to know if you have ever encountered such a problem and how to solve it, or could you give me some advice on what might be the problem.
Thanks for your reading and I look forward to hearing from you.

Hi
Can you please explain me the purpose of data.py in voc12 folder?

Also can we be provided with the scripts that are used to create the .npy files in voc12 folder?