Project Page | ArXiv | Paper
We present GS-IR that models a scene as a set of 3D Gaussians to achieve physically-based rendering and state-ofthe-art decomposition results for both objects and scenes.
Welcome to our new work Analytic-Splatting. We achieve anti-aliasing and excellent detail fidelity through analytical integral approximation.
create the basic environment
conda env create --file environment.yml
conda activate gsir
pip install -r requirements.txtinstall some extensions
cd gs-ir && python setup.py develop && cd ..We evaluate our method on TensoIR-Synthetic and Mip-NeRF 360 datasets. And please visit here for the environment maps. Please refer to the TensoIR for more details about TensoIR-Synthetic and Environment map.
Take the lego case as an example.
Stage1 (Initial Stage)
python train.py \
-m outputs/lego/ \
-s datasets/TensoIR/lego/ \
--iterations 30000 \
--evalStage1 Evaluation (without PBR features)
python render.py \
-m outputs/lego \
-s datasets/TensoIR/lego/ \
--checkpoint outputs/lego/chkpnt30000.pth \
--eval \
--skip_train
Baking
python baking.py \
-m outputs/lego/ \
--checkpoint outputs/lego/chkpnt30000.pth \
--bound 1.5 \
--occlu_res 128 \
--occlusion 0.25Stage2 (Decomposition Stage)
python train.py \
-m outputs/lego/ \
-s datasets/TensoIR/lego/ \
--start_checkpoint outputs/lego/chkpnt30000.pth \
--iterations 35000 \
--eval \
--gamma \
--indirectset
--gammato enable linear_to_sRGB will cause better relighting results but worse novel view synthesis results set--indirectto enable indirect illumination modelling
Evaluation (Novel View Synthesis)
python render.py \
-m outputs/lego \
-s datasets/TensoIR/lego/ \
--checkpoint outputs/lego/chkpnt35000.pth \
--eval \
--skip_train \
--pbr \
--gamma \
--indirectEvaluation (Normal)
python normal_eval.py \
--gt_dir datasets/TensoIR/lego/ \
--output_dir outputs/lego/test/ours_NoneEvaluation (Albedo)
python render.py \
-m outputs/lego \
-s datasets/TensoIR/lego/ \
--checkpoint outputs/lego/chkpnt35000.pth \
--eval \
--skip_train \
--brdf_evalRelighting
python relight.py \
-m outputs/lego \
-s datasets/TensoIR/lego/ \
--checkpoint outputs/lego/chkpnt35000.pth \
--hdri datasets/TensoIR/Environment_Maps/high_res_envmaps_2k/bridge.hdr \
--eval \
--gammaset
--gammato enable linear_to_sRGB will cause better relighting results but worse novel view synthesis results
Relighting Evaluation
python relight_eval.py \
--output_dir outputs/lego/test/ours_None/relight/ \
--gt_dir datasets/TensoIR/lego/Take the bicycle case as an example.
Stage1 (Initial Stage)
python train.py \
-m outputs/bicycle/ \
-s datasets/nerf_real_360/bicycle/ \
--iterations 30000 \
-i images_4 \
-r 1 \
--eval
-i images_4for outdoor scenes and-i images_2for indoor scenes-r 1for resolution scaling (not rescale)
Baking
python baking.py \
-m outputs/bicycle/ \
--checkpoint outputs/bicycle/chkpnt30000.pth \
--bound 16.0 \
--occlu_res 256 \
--occlusion 0.4Stage2 (Decomposition Stage)
python train.py \
-m outputs/bicycle \
-s datasets/nerf_real_360/bicycle/ \
--start_checkpoint outputs/bicycle/chkpnt30000.pth \
--iterations 40000 \
-i images_4 \
-r 1 \
--eval \
--metallic \
--indirectset
--metallicchoose to reconstruct metallicness set--gammato enable linear_to_sRGB will cause better relighting results but worse novel view synthesis results set--indirectto enable indirect illumination modelling
Evaluation
python render.py \
-m outputs/bicycle \
-s datasets/nerf_real_360/bicycle/ \
--checkpoint outputs/bicycle/chkpnt40000.pth \
-i images_4 \
-r 1 \
--eval \
--skip_train \
--pbr \
--metallic \
--indirectset
--gammato enable linear_to_sRGB will cause better relighting results but worse novel view synthesis results
Relighting
python relight.py \
-m outputs/bicycle \
-s datasets/nerf_real_360/bicycle/ \
--checkpoint outputs/bicycle/chkpnt40000.pth \
--hdri datasets/TensoIR/Environment_Maps/high_res_envmaps_2k/bridge.hdr \
--eval \
--gammaset
--gammato enable linear_to_sRGB will cause better relighting results but worse novel view synthesis results
If you find this work useful in your research, please cite:
@article{liang2023gs,
title={Gs-ir: 3d gaussian splatting for inverse rendering},
author={Liang, Zhihao and Zhang, Qi and Feng, Ying and Shan, Ying and Jia, Kui},
journal={arXiv preprint arXiv:2311.16473},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent