NDC space and depth loss about dsnerf HOT 5 CLOSED

Hi! Thanks for your interest!

If you are using NDC space, you need to figure out the linear transformation to NDC space, which is about the near/far plane, and apply the transformation to the depths.

from dsnerf.

Can you please open up your answer a little bit more?

I project the depth values into the image space to get the depth values and coordinates for this depth value for get_rays_by_coord_np.

My bds.min() and bds.max() is 3.5933347725766653 56.21810500817462

What do you mean by figuring out the linear transformation?

Also, we should convert the poses into [r, u, -t] and my
perspective cameras are in x = right, y = down, z = forward coordinate system
my world coordinate system is : x = forward, y = left, z = up.

I converted them into [r,u, -t] by saying
poses = np.concatenate([poses[:, 0:1, :], -poses[:, 1:2, :], -poses[:, 2:3, :], poses[:, 3:, :]], 1)

Does my calculations make sense?

Thank you so much :)

from dsnerf.

Hey again,

Can you please explain how can I convert the depth map in NDC into real depths to compare for the loss? I could not manage to do it.

from dsnerf.

I think by "figure out the linear transformation to NDC space, which is about the near/far plane, and apply the transformation to the depths", the author probably means you should convert depth as

ndc_depth = (depth - near) / (far - near)

from dsnerf.

I was wrong above. The conversion to NDC space is not so simple. Please go through the NDC derivation given by Mildenhall here. The transformation I found was

ndc_depth = 1 - (oz + tn * dz) / (oz + depth * dz)

Again, this is what I think should be the correct mapping, though I'm not 100% sure.

from dsnerf.