Giter VIP home page Giter VIP logo

pandora's Introduction

PANDORA: Polarization-Aided Neural Decomposition Of Radiance


Official PyTorch implementation of Pandora: an inverse rendering technique that exploits neural implicit representations and polarization cues.

PANDORA: Polarization-Aided Neural Decomposition Of Radiance

Akshat Dave, Yongyi Zhao, Ashok Veeraraghavan

Computational Imaging Lab, Rice University

accepted for ECCV 2022

Teaser Animation

Setting up

Loading conda environment

Create a new Anaconda environment using the supplied environment.yml

conda env create -f environment.yml

Downloading datasets

Unzip this zip file (4.5 GB) into the data folder of the repo directory. The zip file contains real and rendered multi-view polarimetric datasets shown in the paper.

Refer to dataio/Ours.py and data/Mitsuba2.py for pre-processing of real and rendered data respectively.

Training

Run the following command to train geometry and radiance neural representations from multi-view polarimetric images.

python -m train --config configs/real_ceramic_owl.yaml

Config files input through --config describe the parameters required for training. As an example the parameters for real ceramic owl dataset are described in real_ceramic_owl.yaml

Tensorboard logs, checkpoints, arguments and images are saved in the corresponding experiment folder in logs/.

Rendering Trained Representations

Using the saved arguments from config.yaml and the saved checkpoint such as latest.pt in logs/, novel views can be rendered using the following command.

python -m tools.render_view 
    --config  logs/our_ceramic_owl_v2/config.yaml
    --load_pt logs/our_ceramic_owl_v2/ckpts/latest.pt

Refer to this documentation in neurecon repo for possible camera trajectories. By default first three views used for training are rendered.

Outputs are saved in the corresponding experiment folder in out/. By default,the outputs include surface normal, diffuse radiance, specular radiance and combined radiance for each view along with the estimated roughness.

Acknowledgements

This repository adapts code or draws inspiration from

Citation

@article{dave2022pandora,
  title={PANDORA: Polarization-Aided Neural Decomposition Of Radiance},
  author={Dave, Akshat and Zhao, Yongyi and Veeraraghavan, Ashok},
  journal={arXiv preprint arXiv:2203.13458},
  year={2022}
}

pandora's People

Contributors

yyiz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

pandora's Issues

how were hdr files obtained

The work is pretty excellent!
However, when I tried running the model on my own data (raw files), I found that the model got stokes vector through HDR files named "s0p1", "s0p2". And I didn't find how to get them from the raw data.

Run on custom data

I tried to train your method on my own data, and I have got some meaningful results. But unfortunately, the default parameters are not the best choice for every scenario, so I'd like to ask what hyperparameters I can tune to get the best results for different data? Thanks for sharing your experience. @akshatdave @yyiz

camera. npz

Thank you for your excellent work! and, may i ask how to obtain camera. npz?

Config file for the render_dataset

Hello
Excellent works
Very useful for my research
I would like to know if you could send us the config file to train on the render_dataset Mitsuba2
Thanks in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.