Run get_rep.py about dery HOT 7 OPEN

Dear @JackqqWang,

Thank you for your interest. The get_rep.py script typically processes the configs for pre-trained models, computes the feature embeddings, and saves them as a .pth file.

You can find all the configurations for my experiments at https://github.com/Adamdad/DeRy/tree/main/configs/compute_sim. If you require additional configs, feel free to create your own or import pre-trained models from other repositories.

Best regards.

from dery.

Hi @Adamdad,
I ran:

PYTHONPATH="$PWD" python simlarity/get_rep.py ./configs/compute_sim/resnet50_imagenet.py --out ./Feature/

but it raised error:

FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/meta/val.txt'

and

FileNotFoundError: ImageNet: [Errno 2] No such file or directory: 'data/imagenet/meta/val.txt'

Did I run the code using the wrong config or the wrong way? Or it was the mistakenly use of the dataset?

Thanks.

from dery.

Dear @Aris-z,

Thank you so much for expressing interest in our work.

It seems that you are encountering an ImageNet version issue. In older versions of ImageNet, data split files for train.txt and val.txt were provided, which the mmclassification package follows:
https://mmclassification.readthedocs.io/en/latest/tutorials/new_dataset.html

However, in some of the latest versions, the data split is no longer specified using two txt files, but instead by employing a list of folders with class names.

To resolve this issue, you can simply set the appropriate configuration to none in the config, which should then load the data according to the folder names.

ann_file=None

I hope this helps! If you have any further questions or concerns, please feel free to reach out.

Best regards.

from dery.

Hi @Adamdad,
Thanks for your work. I encountered the same error as the one above when running get_rep.py, and then I modified the configuration file according to your answer: ann_file=None in configs/base/datasets/imagenet_bs64.py, but the following error was reported:
FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/val' and FileNotFoundError: ImageNet: [Errno 2] No such file or directory: 'data/imagenet/val'.
Is this because I made a mistake in modifying the configuration file or is there another problem? I also hope you can give me answers.
Thanks.

from dery.

Dear @xuelexuebuhui
I believe that is the problem associate with mmcls data loader. Please see you imagenet dataset folder and make sure the subfolder of data/imagenet/val is indeed there.

Best

from dery.

indeed so! Thank you very much for clearing up my confusion.
But I don't know if I need to download this data set myself? If I need to download it myself, which folder should I put it in? And do I need to make changes to it? If there are other data sets that need to be downloaded, is there a way to download them all at once?
Sorry to bother you again.

from dery.

Dear @xuelexuebuhui ,
We mainly use imagenet to extract feature and comparing models. You need to download it yourself. There are a lot of guideline/tutorial for downloading this very well-known dataset.

Best

from dery.