image-level上的pretrain model到底train的哪部分 about hake-action HOT 7 CLOSED

The pretrain model (In table Model Zoo) is only trained on human-object interaction classification and the pasta classification part is not involved. However, model zoos like, PaStaNet*-Linear are trained with both hoi classifcation and pasta classification. You can load these models to test.

from hake-action.

It confused me. From the paper, I thought pretrain is used for activity2vec, which means the pasta classification, and PaSta linear is used in finetuning a2v and pasta r. Isn’t it?

from hake-action.

Sorry for the confusion, but in this repo, the pretrain model is only trained on the HOI classification task. The models except for the Pretrain-model in Table model zoo are trained with pasta classification. You can use these models to initialize the parameters if you need.

from hake-action.

OK. I see. So which dataset is used for pretrain. As I know, the PaStaNet dataset dose not have the HOI label. So you provide the HOI label for the PaStaNet dataset? BTW, can you provide the PaStaNet dataset or dataset used for the pretrain? Thanks.

from hake-action.

Hi, I test the pretrained model using HICO test data, the HOI accuracy of the language part is 0 and all output are around 0.5, which seems the last layer is not trained. So, Is the HOI class of the pretrained model different from the 600 HICO class?

from hake-action.

我又看了一下论文，我大概理解了，这里预训练是分类156个动作。但是还有一个疑问就是，你们提供的pretrain model是在118K PaStaNet全部数据上（除掉HICO test）训练的，还是38K HICO train数据，这里没做说明。

然后就是如果对于PaStaNet*-Linear，是不是就不需要预训练了，因为预训练的话并没有用额外数据。

from hake-action.

PaStaNet*-x: we use the images in HICO train set and their PaSta labels. The only additional supervision here is the PaSta annotations compared to the conventional way.
Please refer to our paper (sec. 5.2) and HAKE-Data repo.

from hake-action.