sinhasam / vaal Goto Github PK

Hello, your paper mentioned that the models begin with an initial labeled pool with 10% of the training set labeled, but in your code you use 5% as the initial proportion.

I am not sure whether the initial proportions in your paper and code are different or I misunderstand the "initial labeled pool".

Thanks.

Dear authors,
Thank you for sharing the code. I recently run the code on CIFAR10 dataset without changing any settings in the code. I achieved only 57.3% accuracy (44.0, 48.0, 52.7, 49.9, 41.6, 59.9, 57.3 for each cycle), which is far from 80.9% accuracy reported in the paper.

It seems that the settings in the code are not the final version. It will be very kind of you if you could provide the settings.

Best Wishes~

I tried to replicate your results in your paper for cifar 10 with the help of your code.I found the following things in paper and code contradicting , can you help me out please?
Issues:

1. In paper you mentioned for cifar 10, you trained with 100 epochs. But code was with respect to iterations.
   I resolved it as for budget b,
   no. of iterations = (b*( no. of epochs))/(batch size)
   So number of iterations grows as split size increases.

2. If I remove the Vaal component, and perform random Sampling, I am not getting the results of your paper.
   I am getting (63,70,76,79,79,81) accuracies for each split which are on par when I ran using Vaal. and quite high compared to your paper.

Hello,
When I reproduce the results on Caltech-256, I found that nearly no classification model can reach 90% Acc-rate on that dataset and 85% Acc-rate is also very difficult with special data augmentation algorithm and very deep network. Can you release your code for Caltech-256?
Thank you.

I tried to implement the vaal on cityscapes.
The setting of experiment is:
resolution: 256 * 256
VAE laten dim(z): 128
lr: 5e-4
Have you met the same condition？

Why is the predictive value of discriminator always around 0.5, and the adv_loss of VAE and D cannot be reduced?
Hope to get your reply.
Thanks a lot.

Is it intended to not convert VAE and Discriminator to eval mode when sampling? For Discriminator it doesnt seem to matter since it is a simple MLP but VAE contain batchnorm layers.

Hi,
I really appreciate that you shared your code online.
I tried testing out your code on CIFAR10 and got similar performance to the paper, but after testing on CIFAR100, the reported scores are quite far off. I am getting around 20% accuracy with 10% data, and only around 39% accuracy at 40% data.
I thought it could be the instability of the VAAL, so I tested random sampling and I seem to be getting similar performance here.

Is there parameters I need to change or that I am missing?

Thanks in advance

Hi @sinhasam

Thank you for shared code. I am looking your paper and released code. You mention the code for segmentation also be released. But I fail to get it. Do you plan to share it or not? If it is impossible, can you introduce the detailed information about the WAE architecture for segmentation. I am wondering whether it is same model with the one for cifar and Imagenet.

Thanks

I am looking for the implementation of your caltech-256 dataset and image segmentation benchmarks, other you provided in the paper.

I can see only implementation of only CIFAR 10,100,Imagenet regular experiments here in the repo.

Can you provide links for that code? Is it publicly available yet?

Hyper-parameters like batch_size, number of epochs and learning rate

Hi, thanks for your great work! However, I am confused about how to run VAAL on Cityscapes (a semantic segmentation dataset). Could you release codes for segmentation?

Hi

Thanks for your shared code. I am trying to run your code ,but I got some problems. When my VAE input shape is [64, 3, 224, 224], I will get the recon's shape is [3136, 3, 32, 32]. I think it might be a problem with VAE parameters

Hello.

Thanks for your great work based on representation. And this method does save more time than Emsemble, Core-set, or MC Dropout when querying informative samples.

However, I got some questions.

• I find it quite time-consuming when using transductive representation learning, which uses all the images in the trainset. That can be a duplicate process when training on the next stages (15%, 20%, ... 40%) since the majority of labelset and unlabelset of the trainset hasn't changed.
• And if we train 100 epochs for VAE and Discriminator, the task model has been trained more than 100 epochs since len(labelset) < len(trainset). I think splitting the task model from the VAE and Discriminator may be a better choice.

CIFAR10 Train Process

python main.py --cuda --dataset cifar10  --data_path /nfs/xs/Datasets/CIFAR10 \
--batch_size 128 --train_epochs 100 \
--latent_dim 32 --beta 1 --adversary_param 1

Iter 1000/39062, task loss: 0.213, vae: 3.931, dsc: 1.389:   3%|█ | 1046/39062 [21:38<13:05:31, 1.24s/it]

Sincerely

I'm looking forward to your advice.
Thanks a lot.

Hi,
Thanks for your great work. I have two questions:

1. the batch size of labeled/unlabelled dataloader is the same, while the total number of labeled/unlabeled data set is different. That is, when one type of data is all trained, the other one still has data left. Am I right?
2. In your code, part of unlabelled data is used to train VAE and Discriminator, the sampler still select uncertain data from all the unlabelled data set (including those parts for training VAE and Discriminator). Most likely the sampler will choose the trained unlabelled data.

Thank you for your time.

Hi,
Did you implement Core-set in pytorch in order to check its performance? If so, can you please share your implementation? here/ privately
Thank you,
Amit

Hello,

In the code for VAE the input is 32x32 (CIFAR 10/100) and after the 4 conv layers it reduces to 4x4 (This is also mentioned in comments in the code).

However, in the very next line while flattening the tensor for the fully-connected layer you use View(-1, 1024*2*2), whereas it should be View(-1, 1024*4*4).

Can you kindly look into this?

Hello,

I have some questions about:
In the code Solver.py->def train
self.args.train_iterations = (self.args.num_images * self.args.train_epochs) // self.args.batch_size

However, in each cycle, for example, from budget 5000 to 7500, the self.args.num_images is always 50000 for cifar10, should not it be the number of current labeled data number like 7500? Because in each cycle, the train data is only the labeled data instead of the whole dataset with size 50000.

Thank you

Congrats for nice paper, it was a very well read! I have doubt in function test of solver class i.e while computing the test accuracy I do not observe the task_model in eval mode. Now as the task_model is vgg16_bn which has batchnorm and dropout layers, so it is must to make the task_model in eval mode before testing. Kindly let me know, if I am wrong somewhere in understanding your code.

Hi Madam,

Thank you so much for sharing your code and work.
I was wondering if your code helps me with Resnet18 architecture. I want to learn the same model for ResNet18

Hi,
I was wondering why did you use a VAE to get the embedding instead of just the output of a middle layer from a CNN?

how long it will take to retrain all the models? I am running main.py for more than one week.

Is there any GPU implementation of main.py ?

Hello, could you answer these questions:

1. Why there are no separate val/train transformers? For example, does ImageNet need to replace random transforms with these: transforms.Resize(256), transforms.CenterCrop(224)?
2. What's the VAE architecture for ImageNet with 224x224 inputs? Currently, only 32x32 is given
3. Do you plan to release code for segmentation task?

Is the epoch in your paper the same as train_iterations in your code? I find that the default value of train_iterations in your code is 100000. Should I change it to 100? By the way, how long does it take for you to run a cifar10 experiment in your environment?

Hi sinha
When I run your code( cifar 10 for example), the trainning time is too long(GTX 1080), which takes about 15 seconds in an iteration(the default number is 100000!) . How to deal with such issue? Do you terminate the trainning by setting a new number?
Thanks!

Hi Samarth, thanks for sharing your code.
However, I met a problem when I run this code with ImageNet dataset. The VAE class in model.py hard code the sample shape of z as 1024*4*4, which is not in line with the image size 224*224 of Imagenet. So, I would like to know whether I should resize the image to 32*32 or I should modify the VAE architecture? Thanks.

Hello,

Very interesting paper and very well-written code. My question might seem dumb. I was wondering if there is a reason not to use bias for conv2dtranspose layers? Is this some kind of known technique?

https://github.com/sinhasam/vaal/blob/master/model.py#L43

Thank you,
Siavash

Hi, thanks for your interesting work. I found your provided code can give the similar performance in your paper on CIFAR10. However, this implementation gives only 5%-10% accuracy on CIFAR100 test set when training with 5000 seed set. Could your please give me some help to handle this problem? Thanks.

Moreover, I read your implementation carefully and found there is no normalization in your dataloader? Why? Thanks again.