AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognition

pytorch==1.6

Under the "code" forder:

• NTU-60
  • Download the NTU-60 data from the https://github.com/shahroudy/NTURGB-D to ../data/raw/ntu60
  • Process the raw data sequentially with python prepare/ntu60/get_raw_skes_data.py, python prepare/ntu60/get_raw_denoised_data.py and python prepare/ntu60/seq_transformation.py
• NTU-120
  • Download the NTU-120 data from the https://github.com/shahroudy/NTURGB-D to ../data/raw/ntu120
  • Process the raw data sequentially with python prepare/ntu120/get_raw_skes_data.py, python prepare/ntu120/get_raw_denoised_data.py and python prepare/ntu120/seq_transformation.py
• SHREC
  • Download the SHREC data from http://www-rech.telecom-lille.fr/shrec2017-hand/ to ../data/raw/shrec_hand
  • Generate the train/test splits with python prepare/shrec/gendata.py

First, pre-train the single-models by:

`python train.py --config ./config/ntu60/ntu60_singlesgn.yaml`

Modify the "gcn_type" and the "num_joint" of the config file to obtain different single-models.

Second, modify the single model paths ("pre_trains" option) in the config file and train the AdaSGN by:

`python train.py --config ./config/ntu60/ntu60_ada_pre.yaml`

Repeat the above two steps to train the bone modality and the velocity modality. In detail, set "decouple_spatial" to True for the bone modality and set "num_skip_frame" to 1 for the velocity modality. Then combine the generated scores with:

`python ensemble.py --label label_path --joint joint_score_path --bone bone_score_path --vel vel_score_path`

Please cite the following paper if you use this repository in your reseach.

@inproceedings{adasgn2021arxiv,  
      title     = {AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognition},  
      author    = {Lei Shi and Yifan Zhang and Jian Cheng and Hanqing Lu},  
      booktitle = {ArXiv:2103.11770 [cs.CV]},  
      year      = {2021},  
}

For any questions, feel free to contact: [email protected]