We track weekly NeRF papers and classify them. All previous published NeRF papers have been added to the list. We provide an English version and a Chinese version. We welcome contributions and corrections via PR.

We also provide an excel version (the meta data) of all NeRF papers, you can add your own comments or make your own paper analysis tools based on the structured meta data.

The Block-NeRF builds the largest neural scene representation to date, capable of rendering an entire neighborhood of San Francisco. The abstract of the Block-NeRF paper is as follows:

We present Block-NeRF, a variant of Neural Radiance Fields that can represent large-scale environments. Specifically, we demonstrate that when scaling NeRF to render city-scale scenes spanning multiple blocks, it is vital to decompose the scene into individually trained NeRFs. This decomposition decouples rendering time from scene size, enables rendering to scale to arbitrarily large environments, and allows per-block updates of the environment. We adopt several architectural changes to make NeRF robust to data captured over months under different environmental conditions. We add appearance embeddings, learned pose refinement, and controllable exposure to each individual NeRF, and introduce a procedure for aligning appearance between adjacent NeRFs so that they can be seamlessly combined. We build a grid of Block-NeRFs from 2.8 million images to create the largest neural scene representation to date, capable of rendering an entire neighborhood of San Francisco.

The official results of Block-NeRF:

This project is the non-official implementation of Block-NeRF. You are expected to get the following results in this repository:

1. Large-scale NeRF training. The current results are as follows:

2. SOTA custom scenes. Reconstruction SOTA NeRFs based on your collected photos. Here is a reconstructed video of my work station:

3. Google Colab support. Run trained Block-NeRF on Google Colab with detailed visualizations (unfinished yet):

The other features of this project would be:

• PyTorch Implementation. The official Block-NeRF paper uses tensorflow and requires TPUs. However, this implementation only needs PyTorch.

• GPU efficient. We ensure that almost all our experiments can be carried on eight NVIDIA 2080Ti GPUs.

• Quick download. We host many datasets on Google drive so that downloading becomes much faster.

• Uniform data format. The original Block-NeRF paper requires downloading tons of data from Google Cloud Platform. This repo provide processed data and convenient scripts. We provides a uniform data format that suits many datasets of large-scale neural fields.

• State-of-the-art performance. This project produces state-of-the-art rendering quality with better efficiency.

• Quick validation. We provide quick validation tools to evaluate your ideas so that you don't need to train on the full Block-NeRF dataset.

• Open research. Along with this project, we aim to developping a strong community working on this. We welcome you to joining us (if you have a Wechat, feel free to add my Wechat ytc407). The contributors of this project are listed at the bottom of this page!

• Chinese community. We will host regular Chinese tutorials and provide hands-on videos on general NeRF and building your custom NeRFs in the wild and in the city. Welcome to add my Wechat if you have a Wechat.

Welcome to star and watch this project, thank you very much!

• [2022.8.24] Support the full Mega-NeRF pipeline.
• [2022.8.18] Support all previous papers in weekly classified NeRF.
• [2022.8.17] Support classification in weekly NeRF.
• [2022.8.16] Support evaluation scripts and data format standard. Getting some results.
• [2022.8.13] Add estimated camera pose and release a better dataset.
• [2022.8.12] Add weekly NeRF functions.
• [2022.8.8] Add the NeRF reconstruction code and doc for custom purposes.
• [2022.7.28] The data preprocess script is finished.
• [2022.7.20] This project started!

Click the following sub-section titles to expand / collapse steps.

Note we provide useful debugging commands in many scripts. Debug commands require a single GPU card only and may run slower than the standard commands. You can use the standard commands instead for conducting experiments and comparisons. A sample bash file is:

# arguments
ARGUMENTS HERE  # we provide you sampled arguments with explanations and options here.
# for debugging, uncomment the following line when debugging
# DEBUG COMMAND HERE
# for standard training, comment the following line when debugging
STANDARD TRAINING COMMAND HERE

bash data_proprocess/download_waymo.sh  // download waymo datasets
bash data_preprocess/download_mega.sh   // download mega datasets from the CMU server. The total size is around 31G.

bash data_preprocess/process_mega.sh

bash scripts/eval_trained_models.sh
# The rendered images would be placed under ${EXP_FOLDER}, which is set to data/mega/${DATASET_NAME}/exp_logs by default.

bash scripts/create_cluster_mask.sh
# The output would be placed under the ${MASK_PATH}, which is set to data/mega/${DATASET_NAME}/building-pixsfm-grid-8 by default.

bash scripts/train_sub_modules.sh SUBMODULE_INDEX # SUBMODULE_INDEX is the index of the submodule.

bash scripts/merge_sub_modules.sh

You may cite this repo to better convince the reviewers about the reproducibility of your paper. If this repo helps you, please cite it as:

@software{Zhao_PytorchBlockNeRF_2022,
author = {Zhao, Zelin and Jia, Jiaya},
month = {8},
title = {{PytorchBlockNeRF}},
url = {https://github.com/dvlab-research/BlockNeRFPytorch},
version = {0.0.1},
year = {2022}
}

The original paper Block-NeRF and Mega-NeRF can be cited as:

 @InProceedings{Tancik_2022_CVPR,
    author    = {Tancik, Matthew and Casser, Vincent and Yan, Xinchen and Pradhan, Sabeek and Mildenhall, Ben and Srinivasan, Pratul P. and Barron, Jonathan T. and Kretzschmar, Henrik},
    title     = {Block-NeRF: Scalable Large Scene Neural View Synthesis},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {8248-8258}
}

@inproceedings{turki2022mega,
  title={Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs},
  author={Turki, Haithem and Ramanan, Deva and Satyanarayanan, Mahadev},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={12922--12931},
  year={2022}
}

We refer to the code and data from DVGO, Mega-NeRF, and SVOX2, thanks for their great work!

Thanks goes to these wonderful people (emoji key):

Zelin Zhao 💻 🚧

EZ-Yang 💻

This project follows the all-contributors specification. Contributions of any kind welcome!