Giter VIP home page Giter VIP logo

wpfhtl / probabilistic-volumetric-3d-reconstruction Goto Github PK

View Code? Open in Web Editor NEW

This project forked from aliosmanulusoy/probabilistic-volumetric-3d-reconstruction

0.0 2.0 0.0 113.23 MB

Probabilistic Volumetric 3D Reconstruction

Home Page: https://ps.is.tue.mpg.de/research_projects/volumetric-reconstruction

Shell 0.10% CMake 1.37% C++ 70.16% HTML 0.11% Perl 0.39% C 21.23% Makefile 0.19% Python 1.04% Objective-C 0.05% TeX 1.02% PostScript 0.48% CSS 0.02% JavaScript 0.01% MATLAB 0.01% OpenEdge ABL 0.13% Fortran 3.34% Batchfile 0.01% WebAssembly 0.01% Roff 0.37%

probabilistic-volumetric-3d-reconstruction's Introduction

Probabilistic Volumetric 3D Reconstruction

This repository implements a probabilistic and volumetric 3D reconstruction algorithm. The algorithm takes as input images (with known camera pose and intrinsics) and generates a dense probabilistic 3D model that exposes the uncertainty in the reconstruction. Please see the video below for a short explanation and results.

Towards Probabilistic Volumetric Reconstruction using Ray Potentials

If you use this software please cite the following publication:

@inproceedings{3dv2015,
  title = {Towards Probabilistic Volumetric Reconstruction using Ray Potentials},
  author = {Ulusoy, Ali Osman and Geiger, Andreas and Black, Michael J.},
  booktitle = {3D Vision (3DV), 2015 3rd International Conference on},
  pages = {10-18},
  address = {Lyon},
  month = oct,
  year = {2015}
}

Note: The code implements a modified version of the algorithm described in the paper. Namely, the belief update of the appearance variables is exchanged with an online-EM approach originally proposed by Pollard and Mundy [CVPR2007]. Our experiments show that this approximation is much faster compared to the original approach proposed in our paper and produces comparable results.

Contact:

If you run into any issues or have any suggestions, please do not hesitate to contact Ali Osman Ulusoy at [email protected]

Requirements:

Compilation:

cd /path/to/my/build/folder
cmake /path/to/vxl/source/folder -DCMAKE_BUILD_TYPE=Release
make -j -k

If everything compiled correctly you should see the executable /path/to/my/build/folder/bin/boxm2_ocl_render_view as well as the library /path/to/my/build/folder/lib/boxm2_batch.so

Add python scripts to the PYTHONPATH as follows,

export PYTHONPATH=$PYTHONPATH:/path/to/my/build/folder/lib/:/path/to/vxl/source/folder/contrib/brl/bseg/boxm2/pyscripts/

Tutorial

The input

  • Images: The current implementation works with intensity images. If you supply an RGB image, it will be automatically converted to intensity.

  • Cameras: Our algorithm expects camera intrinsics (K 3x3 matrix) and extrinsics ([R|t] 3x4 matrix) for each image. The projection matrix is P = K [R | t]. Cameras are specified in separate text files for each image. Each camera text file is formatted as follows:

f_x    s    c_x
0     f_y   c_y
0      0     1
\n
R_11   R_12  R_13
R_21   R_22  R_23
R_31   R_32  R_33
\n
t_1    t_2   t_3

where f_x and f_y are the focal lenghts, c_x and c_y is the principal point and s is the skew parameter. Together, these parameters make up the intrinsics matrix K. The second matrix is the rotation matrix R. The final row is the translation vector t.

Important note: The reconstruction scripts assume when the list of images and camera files are sorted alphabetically they form a correspondence:

img_files = glob(img_folder + "/*.png")
cam_files = glob(cam_folder + "/*.txt")
img_files.sort()
cam_files.sort()

img = img_files[index]
cam = cam_files[index]

One way to ensure this correspondence is to name the cameras to match the images, i.e., img_00001.png and img_00001.txt.

Scene volume specification

You can specify the dimensions of the volume of interest, minimum allowed voxel size in the octree (in world coordinates), and the prior on occupancy probability (see paper reference above) in an XML file scene_info.xml as follows,

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<bwm_info_for_boxm2>
<bbox maxx="1" maxy="1" maxz="1" minx="0" miny="0" minz="0" >
</bbox>
<min_octree_cell_length val="0.001">
</min_octree_cell_length>
<prior_probability val="0.01">
</prior_probability>
</bwm_info_for_boxm2>

Please run the following python script:

import boxm2_create_scene_scripts.py
boxm2_create_scene_scripts.create_scene('/path/to/scene_info.xml','/path/to/scene/')

This script should create the folder /path/to/scene/ and an xml file called scene.xml in it.

Reconstruction

We provide the following python script to reconstruct the scene from the input images and cameras:

/path/to/vxl/source/folder/contrib/brl/bseg/boxm2/pyscripts/reconstruct.py

Please follow the instructions inside the script.

Visualizing the 3D models

You can visualize the volumetric models using the renderer boxm2_ocl_render_view.

/path/to/my/build/folder/bin/boxm2_ocl_render_view -scene /path/to/scene/scene.xml

The renderer computes expected pixel intensities by ray-tracing the probabilistic 3d volume. You can press d to render depth maps and e to render the entropy in depth distributions. Please see the paper for details.

alt tag

How to export the 3d model as a point cloud

The probabilistic volumetric 3D model can also be visualized as a point cloud. We provide the following script that extracts a point cloud from the 3D model and exports it in XYZ format which can be visualized using CloudCompare:

/path/to/vxl/source/folder/contrib/brl/bseg/boxm2/pyscripts/export_point_cloud.py

This script outputs points that correspond to voxel centers. Point with very small probability are filtered for better visualization. The script also outputs the marginal occupancy belief for each point. CloudCompare can be used to visualize these probabilities as shown below: alt tag

Data

As example data, we provide the three aerial datasets used in the publication: link. The folder contains a subfolder for each scene. Each subfolder contains images, cameras, as well as the final octree structure used to produce the results in the paper. A python script to reconstruct the scene is also included for convenience. Please call python reconstruct.py from inside the subfolder.

probabilistic-volumetric-3d-reconstruction's People

Contributors

aliosmanulusoy avatar andyneff avatar billhoffman avatar blowekamp avatar bradking avatar brandonmayer avatar caioc2 avatar daniel-perry avatar decrispell avatar drewgilliam avatar dstoup avatar hjmjohnson avatar kwrobot avatar linus-sherrill avatar mleotta avatar mundyj avatar ozgecanoz avatar pcarr avatar peterminin avatar rfabbri avatar scott-vsi avatar seanm avatar sudomakeinstall avatar thewtex avatar timcootes avatar vishaljain11 avatar yanggehua avatar ydong-vsi avatar yidongvsi avatar yu-da avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.