Confusion about dataset setting and label's defination about groupvit HOT 4 OPEN

Hi @XuMengyaAmy

Thanks for your interest in our work.

(1) The training set should be in webdataset format with image text pairs. The val is just for zero-shot image classification evaluation, so it should be in [image, cls] pairs with webdataset format.
If you would like to evaluate on a custom segmentation dataset, please follow MMSegmentation dataset prepare tutorial

(2) It follows CLIP style here. You may read this paper for more details.

(3) Please follow data convert section to prepare your dataset, for example, gcc3m.

from groupvit.

Thank you for the answer. I have read the CLIP paper, but could not find any information that would address the following question. Regarding the point (2), is this definition of loss based on the assumption that the labels corresponding to samples within each batch are different? Would this loss still be valid if I have a batch size of 128 with only two classes for training?

from groupvit.

Thank you for the answer. I have read the CLIP paper, but could not find any information that would address the following question. Regarding the point (2), is this definition of loss based on the assumption that the labels corresponding to samples within each batch are different? Would this loss still be valid if I have a batch size of 128 with only two classes for training?

Have you resolved the question (2)? I also have a same confusion.

from groupvit.

My understanding is that the CLIP-based loss is effective if the labels (entities) within the training batch are different. And that seems to be the cases with the training datasets used in groupViT. For example the gcc3m has 16K entities (classes, or labels). I had to find it the hard way by running many experiments with limited number of entities. Let me know if you had a different understanding/observation.

from groupvit.