multi_view_reconstruction with depth map about differentiable_volumetric_rendering HOT 5 CLOSED

using the https://github.com/intel-isl/Open3D/blob/master/examples/python/Basic/rgbd_image.ipynb, i visualize the depth maps.
the result for 000020.jpg with different angles is attached below.

original image

point cloud with angle 1

this is the point cloud
https://drive.google.com/file/d/13LZMxohl3a0IdkJvw5c3bUMcVne6hRY1/view?usp=sharing

the above point cloud file can be view with https://www.creators3d.com/online-viewer

from differentiable_volumetric_rendering.

Hi @tharinduk90 , I was now able to have a look at your data.

The depth you provide is very noisy / not accurate. The problem with your visualization above is that you only show the predicted points if you use a single image, and also you show it from the same view as shown in the image. Then, it will always look good! The question is rather how well does the point cloud look if you use all the views, and project them all into the world.

Here is a screenshot of the point cloud if you use all 22 views:

And here the respective ply file (zipped).

As you can see, the depth is so noisy that it does not lead to a consistent model. Hence, if you train with this depth information, it will rather worsen the results than improve them. If you want to improve your results, a good tool to obtain camera parameters as well as depth maps for multi-view images is Colmap.

Good luck with your research !

from differentiable_volumetric_rendering.

@m-niemeyer , thank you very much for the clarification, can u share the script to create the point cloud ? So i can verify my depth map?

from differentiable_volumetric_rendering.

Hi @tharinduk90 , thanks a lot for your message.

I have now created a FAQ file describing the camera matrices and also an example script of how pixels can be projected to the object-centric world space using the matrices and depth values: https://github.com/autonomousvision/differentiable_volumetric_rendering/blob/master/project_pixels_to_world_example.py

I hope this helps. Good luck!

from differentiable_volumetric_rendering.

@m-niemeyer , thank you very much !!!!!!!!!!!!!!

from differentiable_volumetric_rendering.