Pytorch code for our AAAI 2023 paper "Towards Good Practices for Missing Modality Robust Action Recognition".
Standard multi-modal action recognition assumes that the modalities used in the training stage are complete at inference time: (a) → (b). We address the action recognition problem in situations where such assumption is not established,
i.e., when modalities are incomplete at inference time: (a) →
(c). Our goal is to maintain performance in the absence of
any input modality.
$ git clone https://github.com/sangminwoo/ActionMAE.git
$ cd ActionMAE
- Pytorch 1.11.0
- CUDA Toolkit 11.3
- NVIDIA Apex
- Install Pytorch 1.11.0 with the following command.
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch
-
Goto NVIDIA Apex, and follow the instruction.
-
See requirements.txt for all python dependencies, and you can install them using the following command.
$ pip install -r requirements.txt
$ ./train_val_actionmae_multigpu.sh
See/modify configurations in ActionMAE/lib/configs.py
@inproceedings{woo2023towards,
title={Towards Good Practices for Missing Modality Robust Action Recognition},
author={Woo, Sangmin and Lee, Sumin and Park, Yeonju and Nugroho, Muhammad Adi and Kim, Changick},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={37},
number={1},
year={2023}
}
We appreciate much the nicely organized codes developed by MAE and pytorch-image-models. Our codebase is built on them.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent