This is the official repo of EPP-Net and our work Explore Human Parsing Modality for Action Recognition is accepted by CAAI Transactions on Intelligence Technology (CAAI TRIT), 2024.
You can install necessary dependencies by running pip install -r requirements.txt
Then, you need to install torchlight by running pip install -e torchlight
- NTU RGB+D 60 Skeleton dataset from https://rose1.ntu.edu.sg/dataset/actionRecognition/
- NTU RGB+D 120 Skeleton dataset from https://rose1.ntu.edu.sg/dataset/actionRecognition/
- NTU RGB+D 60 Video dataset from https://rose1.ntu.edu.sg/dataset/actionRecognition/
- NTU RGB+D 120 Video dataset from https://rose1.ntu.edu.sg/dataset/actionRecognition/
- Put downloaded skeleton data into the following directory structure:
- data/
- ntu/
- ntu120/
- nturgbd_raw/
- nturgb+d_skeletons
S001C001P001R001A001.skeleton
...
- nturgb+d_skeletons120/
S018C001P008R001A061.skeleton
...
- Extract person frames from the video dataset according to the following project: Extract_NTU_Person
cd ./data/ntu or cd ./data/ntu120
python get_raw_skes_data.py
python get_raw_denoised_data.py
python seq_transformation.py
- cd
./Parsing - Download checkpoints and put it into the
./checkpointsfolder: pth_file
Run:
python gen_parsing.py --input-dir person_frames_path_based_on_Extract_NTU_Person \
--output-dir output_parsing_path \
--model-restore ./checkpoints/final.pth
Example:
python gen_parsing.py --input-dir ./inputs \
--output-dir ./outputs \
--model-restore ./checkpoints/final.pth
you can visual a parsing feature map by python View.py
On the benchmark of XView, using joint modality, run: python Pose_main.py --device 0 1 --config ./config/nturgbd-cross-view/joint.yaml
On the benchmark of XSub, using joint modality, run: python Pose_main.py --device 0 1 --config ./config/nturgbd-cross-subject/joint.yaml
On the benchmark of XSub, using joint modality, run: python Pose_main.py --device 0 1 --config ./config/nturgbd120-cross-subject/joint.yaml
On the benchmark of XSet, using joint modality, run: python Pose_main.py --device 0 1 --config ./config/nturgbd120-cross-set/joint.yaml
On the benchmark of XView, run: python Parsing_main.py recognition -c ./config/nturgbd-cross-view/parsing_train.yaml
On the benchmark of XSub, run: python Parsing_main.py recognition -c ./config/nturgbd-cross-subject/parsing_train.yaml
On the benchmark of XSub, run: python Parsing_main.py recognition -c ./config/nturgbd120-cross-subject/parsing_train.yaml
On the benchmark of XSet, run: python Parsing_main.py recognition -c ./config/nturgbd120-cross-set/parsing_train.yaml
On the NTU120 benchmark of XSub, run:
python ensemble.py --benchmark NTU120XSub --joint_Score ./Pose_Score/ntu120_XSub_joint.pkl --bone_Score ./Pose_Score/ntu120_XSub_bone.pkl --jointmotion_Score ./Pose_Score/ntu120_XSub_jointmotion.pkl --bonemotion_Score ./Pose_Score/ntu120_XSub_bonemotion.pkl --parsing_Score ./Parsing_Score/NTU120_XSub.pkl --val_sample ./Val_Sample/NTU120_XSub_Val.txt
On the NTU120 benchmark of XSet, run:
python ensemble.py --benchmark NTU120XSet --joint_Score ./Pose_Score/ntu120_XSet_joint.pkl --bone_Score ./Pose_Score/ntu120_XSet_bone.pkl --jointmotion_Score ./Pose_Score/ntu120_XSet_jointmotion.pkl --bonemotion_Score ./Pose_Score/ntu120_XSet_bonemotion.pkl --parsing_Score ./Parsing_Score/NTU120_XSet.pkl --val_sample ./Val_Sample/NTU120_XSet_Val.txt
On the NTU60 benchmark of XSub, run:
python ensemble.py --benchmark NTU60XSub --joint_Score ./Pose_Score/ntu60_XSub_joint.pkl --bone_Score ./Pose_Score/ntu60_XSub_bone.pkl --jointmotion_Score ./Pose_Score/ntu60_XSub_jointmotion.pkl --bonemotion_Score ./Pose_Score/ntu60_XSub_bonemotion.pkl --parsing_Score ./Parsing_Score/NTU60_XSub.pkl --val_sample ./Val_Sample/NTU60_XSub_Val.txt
On the NTU60 benchmark of XView, run:
python ensemble.py --benchmark NTU60XView --joint_Score ./Pose_Score/ntu60_XView_joint.pkl --bone_Score ./Pose_Score/ntu60_XView_bone.pkl --jointmotion_Score ./Pose_Score/ntu60_XView_jointmotion.pkl --bonemotion_Score ./Pose_Score/ntu60_XView_bonemotion.pkl --parsing_Score ./Parsing_Score/NTU60_XView.pkl --val_sample ./Val_Sample/NTU60_XView_Val.txt
For any questions, feel free to contact: [email protected]
@inproceedings{liu2023explore,
author={Liu, Jinfu and Ding, Runwei and Wen, Yuhang and Dai, Nan and Meng, Fanyang and Zhao, Shen and Liu, Mengyuan},
title={Explore Human Parsing Modality for Action Recognition},
booktitle={CAAI Transactions on Intelligence Technology (CAAI TIT)},
year={2024}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent