Giter VIP home page Giter VIP logo

view_prior_learning's Introduction

Learning View Priors for Single-view 3D Reconstruction (CVPR 2019)

This is code for a paper Learning View Priors for Single-view 3D Reconstruction by Hiroharu Kato and Tatsuya Harada.

For more details, please visit project page.

Environment

  • This code is tested on Python 2.7.

Testing pretrained models

Download datasets and pretrained models from here and extract them under data directory. This can be done by the following commands.

mkdir data
cd data
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1G5gelwQGniwGgyG92ls_dfc1VtLUiM3s' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1G5gelwQGniwGgyG92ls_dfc1VtLUiM3s" -O dataset.zip && rm -rf /tmp/cookies.txt
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=119D78nZ329J90yTkfSrq4imRuQ8ON5N_' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=119D78nZ329J90yTkfSrq4imRuQ8ON5N_" -O models.zip && rm -rf /tmp/cookies.txt
unzip dataset.zip
unzip models.zip
cd ../

Quantitative evaluation of our best model on ShapeNet dataset is done by the following command.

python ./mesh_reconstruction/test.py -ds shapenet -nt 0 -eid shapenet_multi_color_nv20_uvr_cc_long

This outputs

02691156 0.691549002544
02828884 0.59788288686
02933112 0.720973934558
02958343 0.804359183654
03001627 0.603543199669
03211117 0.593105481352
03636649 0.502730883482
03691459 0.673864365473
04090263 0.664089877796
04256520 0.654773500288
04379243 0.602735843742
04401088 0.767574659204
04530566 0.616663414002
all 0.653372787125

Other ShapeNet models are listed in test_shapenet.sh.

Drawing animated gif of ShapeNet reconstruction requires the dataset provided by [Kar et al. NIPS 2017].

cd data
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=17GjULuQZsn-s92PQFQSBzezDkonowIxR' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=17GjULuQZsn-s92PQFQSBzezDkonowIxR" -O lsm.tar.gz && rm -rf /tmp/cookies.txt
tar xvzf lsm.tar.gz
cd shapenet_release/renders/
find ./ -name "*.tar.gz" -exec tar xvzf {} \;
cd ../../../

Then, the following commands

mkdir tmp
bash make_gif.sh

output the following images.

Training

Training requires pre-trained AlexNet model.

cd data
mkdir caffemodel
cd caffemodel
wget http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel
cd ../../

Training of the provided pre-trained models is done by

bash train_shapenet.sh
bash train_pascal.sh

Citation

@InProceedings{kato2019vpl,
    title={Learning View Priors for Single-view 3D Reconstruction},
    author={Hiroharu Kato and Tatsuya Harada},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2019}
}

view_prior_learning's People

Contributors

hiroharu-kato avatar

Stargazers

avatar Sangpil Kim avatar cjeen avatar Hao Huang avatar Stefano Pini avatar namwoo.lee (이남우) avatar lulihua avatar An-zhi WANG avatar avatar Yuanchen Guo avatar Gldstrrbt avatar avatar Arnav Jain avatar YJ Wang avatar Linlin Yang avatar avatar Chief Accelerator avatar Longyong Wu avatar Nori avatar Wei Yizhi avatar Tzu-Mao Li avatar diver avatar avatar DeletedName avatar Jamal Dahbur avatar avatar Markus Mayer avatar avatar Yangming Wen avatar 爱可可-爱生活 avatar Gu Wang avatar Takahiro "Poly" Horikawa avatar Chao Wen avatar Hiro Matsuoka avatar Katsuya Hyodo avatar SunshineAtNoon avatar Xa9aX ツ avatar

Watchers

Gu Wang avatar avatar

Forkers

xcyan diggle1 subodh01006 hyunynim cv-ip spk921

view_prior_learning's Issues

Implementations about the FCShapeDecoder

Hi, could you shed light on the usage of obj_scale, tanh_scale, scaling variables in FCShapeDecoder respectively?

  • Why you multiply the intermediate value with the above scaling factor to get the final vertex coordinates? Is that because to reduce the destruction of noise?
  • How to choose the value of those variables? Is that empirically found useful?

    view_prior_learning/mesh_reconstruction/decoders.py

    Lines 382 to 390 in 6f1afa8

    class FCShapeDecoder(chainer.Chain):
    def __init__(
    self, dim_in=512, scaling=1.0, use_bn=False, fc_connection=False, symmetric=False):
    super(FCShapeDecoder, self).__init__()
    self.grid_size = 16
    self.obj_scale = 0.5
    self.tanh_scale = 1.2
    self.scaling = scaling

I also notice that the different method is used to reconstruct 3D mesh in your another repo mesh_reconstruction. What makes you choose different reconstruction algorithms?

Thank you in advance!

Environment dependency

Hi, Could you please share the requirements.txt of this fantastic project? And how to use the neural renderer, is it the same as instructed in neural mesh render?

Thank you in advance!

Voxelization in chainer

Hi @hiroharu-kato

Really enjoyed reading the VPL paper, thanks for making the code public!
I was trying to port the code to python3 and CUDA. I am a bit doubtful about how the voxelize_sub1 should be done. For example, it is not clear to me that how on ine 14 in voxelization.py, i is used, whereas input to the function is j. Also, when it is written voxel=1 like here does it mean that i'th element of voxel array is set to 1?

Any help in this regard is highly appreciated!

Single view dataset

Hi @hiroharu-kato Thanks for the code. I have stuck up with some questions and it expecting answers. I have understood the reconstructor part which generate mesh using encoder-decoder architecture.
1.) For single view training, does discriminator takes only one viewpoint (out of 20) as correct views and remaining 19 as incorrect?
2.) What will be partitions of dataset for single view training of discriminator?
3.) What does real views mean In view-based training, reconstructed views from
observed viewpoints converge to the real views in a training
dataset by minimizing the reconstruction loss (single view)?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.