Question 3: Different intrinsic parameters after alignment
I've run the example codes for "Rendering depth and color with OpenCV and Numpy" and "Align & Background Removal". After alignment, I noticed that the intrinsic parameters (intrinsics) are different between the two codes:

No alignment code:

intrinsics = frames.profile.as_video_stream_profile().intrinsics
print(intrinsics) # Output: 'cam_K': [379.8869934082031, 0.0, 312.4703369140625, 0.0, 379.8869934082031, 240.33531188964844, 0.0, 0.0, 1.0]

Alignment code:

intrinsics = frames.profile.as_video_stream_profile().intrinsics
print(intrinsics) # Output: 'cam_K': [431.27752685546875, 0.0, 313.7019958496094, 0.0, 430.9690856933594, 240.52565002441406, 0.0, 0.0, 1.0]

Why are the intrinsic parameters different after alignment? Shouldn't they be the same?

from librealsense.

Hi @qidihan The Align & Background Removal script that you refer to, also known as align_depth2color.py, uses 640x360 for its alignment resolution by default. At #11329 a RealSense user created a specially edited version of this script for use on D405 and set the resolution to 640x480 but found that it had problems with the alignment. In the end they settled for using 640x360.

Yes, the default depth unit scale of D405 is 0.01. 9.999999747378752e-05 when input into the RealSense Viewer's depth unit scale option auto-converts it into a value of 0.0001, whilst changing the '05' on the end of the value to '02' is required in order for the Viewer to recognize it as 0.01.

It may be unnecessary to retrieve and output the depth scale, as the default scale for a particular camera will be automatically applied (0.01 for D405 and 0.001 for the rest of the 400 Series range) and will not change unless deliberately changed by user input or by a program instruction that sets the depth scale value.

Each resolution has different intrinsic fx / fy values. The default resolution of D405 for depth and color is 848x480. After checking the intrinsics of my own D405, I wonder if you are seeing the intrinsics for 848x480 when alignment is not being performed, but when the alignment script is run and resolution is set at 640x360, you are seeing the different intrinsics for 640x360 resolution.

from librealsense.

Thank you for your response. I still have some questions regarding the depth image unit.

1. I need to output my depth image with unit mm, but when I use:

depth_image = np.asanyarray(aligned_depth_frame.get_data())
cv2.imwrite((depthpath), (depth_image).astype(np.uint16))

I'm unsure what unit the output depth_image.max() value represents. Could you please clarify what unit this value is in? For example, it outputs 65536, which I guess to be equivalent to 655.36mm (how it relates to 0.01 depth scales?). However, I'm unsure how to relate this to the unit millimeter.
My main question is: can I directly output the depth image with unit millimeter by using cv2.imwrite((depthpath), (depth_image).astype(np.uint16)), or do I need to multiply the depth values by 100 to convert them to millimeters?
(I'm working on the BundleSDF project ((https://github.com/NVlabs/BundleSDF)) and need depth images for testing. Thank you for your help and clarification.)

2. Additionally, I have a question regarding the intrinsic parameters. When I use align-depth2color.py and Rendering depth and color with OpenCV and Numpy codes, the intrinsic parameters are different for the same pixel (640*360). However, the output aligned images look perfect. Could you please explain why this is the case?
   The project also needs data like:
   {"cam_K": [614.7604370117188, 0.0, 322.7101135253906, 0.0, 614.9583129882812, 233.1604766845703, 0.0, 0.0, 1.0], "depth_scale": 1.0000000474974513}
   I'm unsure what the depth_scale should be output in my case, if the depth image need unit of millimeter. The example code the author uses is (He is using the D435 camera. not d405):

Get depth profile and camera intrinsics

intrinsics_matrix = {
"cam_K": [
intrinsics.fx,
0.0,
intrinsics.ppx,
0.0,
intrinsics.fy,
intrinsics.ppy,
0.0,
0.0,
1.0,
],
"depth_scale": depth_scale * 1000,
}

from librealsense.

0 to 65536 represents the depth as a pixel depth value. To convert this value to real-world distance, you multiply it by the depth scale of the camera.

For example, if a non-D405 camera was being used then its default scale would be 0.001, so 65535 x 0.001 = would be 65.535 (meters).

However, D405 (which is a short-range camera) uses a scale of 0.01 by default. This is centimeter scale instead of millimeter scale. So 65535 x 0.01 = 655.35 (centimeters), equivalent to 6.5535 meters. So you would multiply the centimeter result by 10 to get the distance in millimeters.

When the align_to instruction is used to perform alignment (like in align_depth2color.py), the RealSense SDK's align processing block performs automatic adjustment calculations.

from librealsense.

Hi @qidihan Do you require further assistance with this case, please? Thanks!

from librealsense.