View Code? Open in Web Editor
NEW
This project forked from pantomatrix/pantomatrix
A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis [ECCV 2022]
JavaScript 6.36%
Python 88.68%
CSS 0.43%
HTML 4.52%
beat's Introduction
BEAT(CaMN) — Official PyTorch implementation
- train and inference scripts
- CaMN (ours)
- End2End (ours)
- Motion AutoEncoder (for evaluation)
- data preprocessing
- load specific number of joints with predefined FPS from bvh
- build word2vec model
- cache generation (.lmdb)
- dataset examples in beat.zip
- original files to generate cache in train/val/test
- cache for language_model, pretrained_vae
python == 3.7- build folders like:
- download the scripts to
codes/beat/
- extract beat.zip to datasets/beat
- run
pip install -r requirements.txt in the path ./codes/beat/
- run
python train.py -c ./configs/camn.yaml for training and inference.
- load
./outputs/exp_name/119/res_000_008.bvh into blender to visualize the test results.
- train End2End model, add
g_name: PoseGenerator in camn.yaml
- generate data cache from stratch
cd ./dataloaders && python bvh2anyjoints.py for motion datacd ./dataloaders && python build_vocab.py for language model
- remove modalities, e.g., remove facial expressions.
- set
facial_rep: None and facial_f: 0 in camn.yaml
python train.py -c ./configs/camn.yaml- for semantic-weighted loss, set
sem_weighted == False in camn_trainer.py
- refer to
./utils/config.py for other parameters.
beat's People
Contributors
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent