baegwangbin / dsine Goto Github PK

Hi, thanks for releasing such great work! I am testing the provided model on the Hypersim.

However, I found the accuracy is not very good. Therefore, I want to have a verification with you: if the bad results are reasonable or there is something wrong in my processing.

I test on the ai_001_010/cam_00 of the Hypersim, and use the frame.xxxx.normal_cam.png as ground truth.
The accuracy is

total_iter (# 78198048): ai_001_010/cam_00:
mean median rmse 5 7.5 11.25 22.5 30
52.066 41.052 69.400 8.312 14.636 26.022 40.487 43.882

Besides, I don't know which camera coordinate system (opencv/opengl) you use and camera in the Hypersim is the opengl coordinate system ( x-axis points right, the positive y-axis points up, and the positive z-axis points away from where the camera is looking.) Therefore, I convert the GT normal to the opencv-camera coordinate system, where I negate the y-axis and z-axis , and evaluate again. The accuracy becomes worse:

total_iter (# 78198048): ai_001_010/cam_00
mean median rmse 5 7.5 11.25 22.5 30
170.977 177.555 171.943 0.000 0.000 0.000 0.001 0.002

Here is a example contains the ground truth, input color image and the results of DSINE.

P.S. I have provide DSINE with the intrinsic of Hypersim.

May I ask how to extract normal vectors from the estimated normal maps which are in rgb format?

Hey guys, I appreciate the awesome work!
when I ran the test with the image from my D405 depth camera, I got this result.
it seems right.

but, when I read the normal from this image, and put it in my point cloud. the normal is not along with the surface. but the orientation of light.

Hello, I have some question about the result normal mapping. Is the normal in tangent space or camera space, and how can I transform results normal map(right) to world space like left.

I was wondering if the coordinate system you describe is the same as the coordinate system used in omnidata v2 which you are comparing against.

Hi, @baegwangbin

Thanks for your awesome work!
I would like to know will the training code be released and if will you train the model on a larger dataset, e.g. Omnidata v2.

Thanks for your great job,The oasis dataset you used in your project is a dataset for single-image 3D in the wild ,those images from different cameras .How did you obtain the camera intrinsic parameters of these images?

Meet error when using torch hub:

I think replacing utils.pad_input with utils.get_padding can solve this error.

Hi, thanks for your great work. I wondered if you plan to open source the training data. Sharing it could benefit the community.

Congrats on the release!!

I wanted to see how this performed so I went ahead and quickly built a huggingface space for it. I wanted to make this was okay before I promoted it. I made sure I added a modal so folks can agree to the license. I also made sure to download the model after the fact such that it doesn't show up in the huggingface space files. Let me know if this is okay!

https://huggingface.co/spaces/pablovela5620/DSINE-space

Hello, thanks for your great work. I'm a student looking into this problem and this repo saved me a month. I wonder if you plan to open source the training code.