For Human3.6M and HumanEva-I:

We have adopted the data preprocessing framework and datasets from GSPS. The datasets can be downloaded from here, and all files should be placed in the ./data directory.

For the zero-shot prediction experiments using the AMASS dataset:

We retargeted the AMASS dataset skeletons to the Human3.6M dataset skeletons. We only provide a subset of the AMASS motions here. The retargeted dataset can be downloaded from Google Drive (Baidu Cloud). Place the .npy files in the ./data directory. Details of the retargeting process can be found in ./motion-retargeting.

The final structure of the ./data directory is as follows:

data
├── amass_retargeted.npy
├── data_3d_h36m.npz
├── data_3d_h36m_test.npz
├── data_3d_humaneva15.npz
├── data_3d_humaneva15_test.npz
├── data_multi_modal
│   ├── data_candi_t_his25_t_pred100_skiprate20.npz
│   └── t_his25_1_thre0.500_t_pred100_thre0.100_filtered_dlow.npz
└── humaneva_multi_modal
    ├── data_candi_t_his15_t_pred60_skiprate15.npz
    └── t_his15_1_thre0.500_t_pred60_thre0.010_index_filterd.npz

For the demonstration of various capabilities of Fast-Human, we provide a pretrained model on Human3.6M available on Google Drive (Baidu Cloud). The pretrained model should be placed in the ./checkpoints directory.

sh install.sh

For Human3.6M:

python main.py --cfg h36m --mode train

For HumanEva-I:

python main.py --cfg humaneva --mode train

After running the command, a directory named <DATASET>_<INDEX> will be created under ./results directory (<DATASET> belongs to {'h36m', 'humaneva'}, <INDEX> equals the number of directories under ./results). During the training process, gifs are stored in ./<DATASET>_<INDEX>/out, log files are saved in ./<DATASET>_<INDEX>/log, model checkpoints are stored in ./<DATASET>_<INDEX>/models, and metrics are stored in ./<DATASET>_<INDEX>/results.

For Human3.6M:

python main.py --cfg h36m --mode pred --vis_row 3 --vis_col 10 --ckpt ./checkpoints/h36m_ckpt.p

For HumanEva-I:

python main.py --cfg humaneva --mode pred --vis_row 3 --vis_col 10 --ckpt ./checkpoints/humaneva_ckpt.pt

vis_row and vis_col represent the number of rows and columns of gifs respectively. Each type of action in the dataset will have two gifs, each gif containing vis_row actions, each action having vis_col candidate predictions. Gifs are stored in ./inference/<DATASET>_<INDEX>/out.

Demonstration of action switching:

python main.py --mode switch --ckpt ./checkpoints/h36m_ckpt.pt

vis_switch_num gifs will be stored in ./inference/switch_<INDEX>/out, each gif containing 30 actions, these actions will eventually switch to one of them.

Demonstration of controllable action prediction:

python main.py --mode control --ckpt ./checkpoints/h36m_ckpt.pt

Seven gifs will be stored in ./inference/<CONTROL>_<INDEX>/out, each gif having vis_row actions, each action having vis_col candidate predictions. <CONTROL> corresponds to {'right_leg', 'left_leg', 'torso', 'left_arm', 'right_arm', 'fix_lower', 'fix_upper'}.

Demonstration of zero-shot testing on the AMASS dataset:

python main.py --mode zero_shot --ckpt ./checkpoints/h36m_ckpt.pt

Gifs of the zero-shot testing experiment will be stored in ./inference/zero_shot_<INDEX>/out, with the number of actions also set by vis_col and vis_row.

For Human3.6M:

python main.py --cfg h36m --mode eval --ckpt ./checkpoints/h36m_ckpt.pt

For HumanEva-I:

python main.py --cfg humaneva --mode eval --ckpt ./checkpoints/humaneva_ckpt.pt

Note: We have parallelized the calculation process of evaluation metrics (APD, ADE, FDE, MMADE, MMFDE) to speed up computation, therefore this part strictly requires the use of a GPU.