Project Page | Video | Paper | Data
Representing Volumetric Videos as Dynamic MLP Maps
Sida Peng*, Yunzhi Yan*, Qing Shuai, Hujun Bao, Xiaowei Zhou (* equal contribution)
CVPR 2023
Any questions or discussions are welcomed!
Please see INSTALL.md for manual installation.
Interactive rendering on ZJU-MoCap
Please see INSTALL.md to download the dataset. We provide the pretrained models at here.
Take the rendering on sequence 313 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/zjumocap/313/final.pth. -
Interactive rendering demo:
python gui.py --config configs/zjumocap/dymap_313.py fast_render True
Interactive rendering on NHR
Please see INSTALL.md to download the dataset. We provide the pretrained models at here.
Take the rendering on sequence sport1 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/nhr/sport1/final.pth. -
Interactive rendering demo:
python gui.py --config configs/nhr/sport1.py fast_render True
Please see INSTALL.md to download the dataset.
We provide the pretrained models at here.
Test on ZJU-MoCap
Take the test on sequence 313 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/zjumocap/313/final.pth. -
Test on unseen views:
python run.py --config configs/zjumocap/dymap_313.py mode evaluate fast_render True
Visualization on ZJU-MoCap
Take the visualization on sequence 313 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/zjumocap/313. -
Visualization:
- Visualize free-viewpoint videos
python run.py --config configs/zjumocap/dymap_313.py mode visualize vis_novel_view True fast_render True
- Visualize novel views of single frame
python run.py --config configs/zjumocap/dymap_313.py mode visualize vis_novel_view True fixed_time True fast_render True
- Visualize the dynamic scene with fixed camera
python run.py --config configs/zjumocap/dymap_313.py mode visualize vis_novel_view True fixed_view True fast_render True
- Visualize mesh
python run.py --config configs/zjumocap/dymap_313.py mode visualize vis_mesh True fast_render True
Training on ZJU-MoCap
Take the training on sequence 313 as an example.
- Train:
# training python train_net.py --config configs/zjumocap/dymap_313.py # distributed training python -m torch.distributed.launch --nproc_per_node=4 train_net.py --config configs/zjumocap/dymap_313.py - Post-process the trained model:
python run.py --config configs/zjumocap/dymap_313.py mode visualize occ_grid True - Tensorboard:
tensorboard --logdir data/record/zjumocap
Please see INSTALL.md to download the dataset.
We provide the pretrained models at here.
Test on NHR
Take the test on sequence sport1 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/nhr/sport1/final.pth. -
Test on unseen views:
python run.py --config configs/nhr/sport1.py mode evaluate fast_render True
Visualization on NHR
Take the visualization on sequence sport1 as an example.
-
Download the corresponding pretrained model and put it to
$ROOT/data/trained_model/nhr/sport1. -
Visualization:
- Visualize novel views
python run.py --config configs/nhr/sport1.py mode visualize vis_novel_view True fast_render True
- Visualize novel views of single frame
python run.py --config configs/nhr/sport1.py mode visualize vis_novel_view True fixed_time True fast_render True
- Visualize the dynamic scene with fixed camera
python run.py --config configs/nhr/sport1.py mode visualize vis_novel_view True fixed_view True fast_render True
- Visualize mesh
python run.py --config configs/nhr/sport1.py mode visualize vis_mesh True fast_render True
Training on NHR
Take the training on sequence sport1 as an example.
- Train:
# training python train_net.py --config configs/nhr/sport1.py # distributed training python -m torch.distributed.launch --nproc_per_node=4 train_net.py --config configs/nhr/sport1.py - Post-process the trained model:
python run.py --config configs/nhr/sport1.py mode visualize occ_grid True - Tensorboard:
tensorboard --logdir data/record/nhr
If you find this code useful for your research, please use the following BibTeX entry.
@inproceedings{peng2023representing,
title={Representing Volumetric Videos as Dynamic MLP Maps},
author={Peng, Sida and Yan, Yunzhi and Shuai, Qing and Bao, Hujun and Zhou, Xiaowei},
booktitle={CVPR},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent