How to modified configs files for tiny or small model about vit-adapter HOT 6 OPEN

Hi, you can set with_cp=True (in backbone and head) to save GPU memory. For example, when the input image is 512x512, the BEiT-Adapter-L-Mask2Former requires about 15G memory with 1 image per card (see my logs in issue #38).

I have provided some configs for ViT-T and ViT-S; you can try these configs and set with_cp=True. Notice that the GPU memory shown in the released logs was inaccurate, due to some bugs in segmentation (getting accurate memory usage requires setting --deterministic).

Also, you can modify cffn_ratio, deform_num_heads, interaction_indexes to reduce GPU memory. cffn_ratio is set to 0.25 now, you can try 0.125. You can try reducing deform_num_heads by half, and reducing interaction_indexes. Of course, these modifications will hurt performance to some extent.

For a 24-layer ViT model, we set N=4, that is to say, interaction_indexes=[[0, 5], [6, 11], [12, 17], [18, 23]]. If you want to set N=2, you should use interaction_indexes=[[0, 11], [12, 23]].

from vit-adapter.

@czczup Hi，i would like to save gpu memory，so i change the crop size from 512 to 256，when training voc2007 dataset，it occurs other issue，assertionerror at the iteration of evaluation（160000/20000）。And i test the voc datasets also occur the same issue。But when i change the crop size from 256 to 512，it tests successfully，but the out of memory issue occurs。 I wonder whether is the reason of change crop size that occurs the assertionerror？I am waiting for your answers，thanks！

from vit-adapter.

@czczup Hi，i would like to save gpu memory，so i change the crop size from 512 to 256，when training voc2007 dataset，it occurs other issue，assertionerror at the iteration of evaluation（160000/20000）。And i test the voc datasets also occur the same issue。But when i change the crop size from 256 to 512，it tests successfully，but the out of memory issue occurs。 I wonder whether is the reason of change crop size that occurs the assertionerror？I am waiting for your answers，thanks！

Hi, in addition to crop size, you also need to set the img_size as same as crop size in backbone. 

from vit-adapter.

@czczup Thanks for your reply.I used to change the crop size in config file and datasets setting.But I can not find the img_size in backbone of vit_adapter.py.If I need to change the img_size in segmentation/mmseg_custom/models/backbones/base/vit.py from 224 to 256,or can you tell in details.thanks for your generous help!

from vit-adapter.

@czczup Thanks for your reply.I used to change the crop size in config file and datasets setting.But I can not find the img_size in backbone of vit_adapter.py.If I need to change the img_size in segmentation/mmseg_custom/models/backbones/base/vit.py from 224 to 256,or can you tell in details.thanks for your generous help!

Could you please provide the config you used.

from vit-adapter.

Regarding your comments on changing layers and interaction indexes, I'm trying to modify these parameters in upernet_deit_adapter_tiny_512_160k_ade20k.py.

Would the red highlight be the number of blocks? (I know N=4 was used in most of the models, but I'm experimenting with changing this. When I make these changes I get the following error:

What other changes might I need to make in order to get this to run?

from vit-adapter.