mwray / joint-part-of-speech-embeddings Goto Github PK

Hi Michael. Thanks for sharing your wonderful work~
I got a few questions for the video feature.
1: I notice that the shape of the video feature from "./data/video_features/EPIC_100_retrieval_{}_features_mean.pkl" is nx3072. Is such a feature obtained by concat 'RGB', 'Flow' and 'Audio' features of size nx25x1024 into nx25x3072 and then average in the time dimension?
2: What model did you use to extract the 'RGB', 'Flow' and 'Audio' features? Is it the TBN model which is trained on EPIC kitchen-100 or EPIC kitchen-55 for action Recognition?

1. The validation split is used for evaluation in your repo. But how is validation split created? I could not find the description about it in your paper.

A problem from the val split is that the annotation dataframes between your provided train and val split have overlapped videos.

>>> tr_annos_jpose = pd.read_pickle(os.path.join(root_path, 'data/dataframes/EPIC_100_retrieval_{}.pkl'.format("train")))
>>> va_annos_jpose = pd.read_pickle(os.path.join(root_path, 'data/dataframes/EPIC_100_retrieval_{}.pkl'.format("validation")))
>>> print(len(tr_annos_jpose), len(va_annos_jpose))
67219 4834
>>> set(va_annos_jpose.index) - set(tr_annos_jpose.index)
{'P22_17_238'}

2. The other question is that why the number of videos in the train split in this repo (67219 as shown above) is different with that in epic-kitchens-100-annotations repo (67217).

Jiankun

Dear Michael, thanks for releasing the code for JPoSE. I have a question about the textual features used for Epic-Kitchens 100.
I would like to use part of scripts/create_feature_files.py but it requires a word2vec model. I tried looking for a wikipedia-pretrained (as is mentioned in the paper) with 200-D vectors but I can not seem to find any. Would it be possible for you to share the one you used?
Thanks,
Alex