In relation to https://github.com/luxonis/depthai-experiments/tree/master/gen2-wls-filter

The disparity color range is jumpy. I don't know if this is out of my ignorance of how WLS works, or if it's something like the following.

Below is a video showing it:
https://youtu.be/IEqg0R2xA3g

With that said, I'm guessing the color range is normalized every frame to the present min and max disparities, or something like that. So perhaps we could do a fixed normalization or some average over time as the scene changes maybe.

Hi @saching13,

I'm using the latest version of depthAI package and trying to explore on your point cloud implementation. There are some interface changes between your Point cloud branch and the master branch. I've merged depthAI changes to master depthAI demo, but I have an issue with calibration changes.

Could you please provide me with the branch name or commit_id that works for you.

Regards,
Jegathesan S

The bounding box (possibly just the UI drawing) of the left and right eyes are not updated according to the right scale. Maybe it is also great to verify we are also passing the data correctly.

note. this issue is mentioned in #51

Recently, I tried to script a gen2 version of social distancing using 26_1_spatial_mobilenet.py as reference to implement #116 (comment). However, using person-detection-retail-0013 OpenVINO v2021.3 model, 7 shaves gives a 10FPS inference.
Recent gen2-people-tracker example notices a 5FPS performance.

Comparing this to Gen1 social distancing which records 29FPS inference on RGB frame, but something similar as well (not recorded) on NN frame. This is a noticable FPS drop, so was wondering if there is a workaround for this. The FPS was recorded for RGB input frame and not the NN frame in gen1 example, but the display output as a 29FPS display, although the bounding boxes record a less FPS.

Hi, I am using the Luxonis Rpi Compute module 3 and am having some trouble with the depthAI.

I replaced the SD card that came with the device with another one and used the Raspbian image provided on the Luxonis website to install the OS. After that I followed the steps on the website https://docs.luxonis.com/projects/api/en/latest/install/ to install the necessary dependencies and the library. It is now currently running depthAI-2.5.0.0.

When I tried running the depthai_demo program I got a runtime error: “Couldn’t read data from stream: ‘__rpc_main’ (X_LINK_ERROR)”. However, when I tried running programs from the depthai-python library, every program that did not use depth worked fine. The RGB video, mono camera, all worked fine but as soon as depth was involved, I got this same error. I was wondering what I could do to fix this problem. Thank you.

About gen2-mask-detection
Your OpenVINO version isn't compatible with my code
So can you share the IR model(.yml, .bin) which i can covert it to Blob model myself?
Many thank <3

Create an application that will detect cards in your hand and on the table and will provide a win probability

Inspiration: https://github.com/EdjeElectronics/OpenCV-Playing-Card-Detector

Hello, I have been running the pipelines successfully from my raspberry pi and would like to run the same pipelines from other rpis. However, installing all the requisites might be time consuming and I would rather create some docker containers to deploy to the rpis.
Has anybody done that already and is there something specific that needs to be done when running from a container? Is there any specifics to setup for accessing the USB port connected to the camera?

Thanks for any hints or experiences.

Marcelo

gen2-deeplabv3_depth experiment is not working with the latest API version (2.7.2.0), the sync mechanism is not updating depth frames even though they arrive.

Potentially solvable by using #159

I tried to migrate this example to latest DepthAI Gen2 API (commit hash 45765d01ad9c7d8d9ae99ca0f76c5b03b706a1c4), and got the following error after running against this version

Traceback (most recent call last):
  File "/home/vandavv/dev/luxonis/depthai-experiments/gen2-color-isp-raw/main.py", line 40, in <module>
    cam.setEnablePreviewStillVideoStreams(False)
AttributeError: 'depthai.ColorCamera' object has no attribute 'setEnablePreviewStillVideoStreams'

This example also lacks a fixed requirements version, README.md and command line parameters

When introduced a parallel implementation of Gaze Estimation example, we started to see the issues with thread termination, leading to unexpected console errors or even crash dialogs to appear.

Examples:

File "main.py", line 312, in
Main().run()
File "main.py", line 302, in run
if cv2.waitKey(1) == ord('q'):
KeyboardInterrupt
Assertion failed: (pthread_mutexAssertion failed: (pthread_mutex_destroy(mutex) == 0), function _lock(mutex) == 0), function usbusbi_mutex_destroy, file ./os/thi_mutex_lock, file ./os/threads_reads_posix.h, line 58.
posix.h, line 46.
zsh: abort python3 main.py -cam

... FYI when I use the 'q' to exit the application. My quick suggestion is to properly join the threads and exit them when user pressed 'q' or ctrl+c.

python3: io.c:2115: handle_events: Assertion `ctx->pollfds_cnt >= internal_nfds' failed.
Aborted (core dumped)

When the time permits, we should investigate these error messages and handle them properly, as it can also affect other examples

Create a mechanism to automatically point a nerf gun turret to a detected object

Inspiration - https://www.hackster.io/hackershack/raspberry-pi-motion-tracking-gun-turret-77fb0b

The Gen2 NN Sync inferencing basically freezes if the face_frame is none, because the loop gets stuck on the place if there is no face detected in the first frame -

            while q_land.has():
                face_frame = face_q.get()
                out = frame_norm(face_frame, q_land.get().getFirstLayerFp16())
                cv2.circle(face_frame, tuple(out[:2]), 1, (255, 255, 0))  # Right eye
                cv2.circle(face_frame, tuple(out[2:4]), 1, (255, 255, 0))  # Left eye
                cv2.circle(face_frame, tuple(out[4:6]), 1, (255, 255, 0))  # Nose
                cv2.circle(face_frame, tuple(out[6:8]), 1, (255, 255, 0))  # Right mouth
                cv2.circle(face_frame, tuple(out[8:]), 1, (255, 255, 0))  # Left mouth
                cv2.imshow("face", face_frame)

An alternative could be using threading, so even though face_frame is not extracted, the main loop could continue to work.
@VanDavv

Hi All;

When I run the Point Cloud Projection example, I get the folloiwng error:

Aftrwarrds, when I running any other demo, including ones in the depthai-python 'examples' folder I get this error:

After some trouble-shooting, what's going on is something in the OAK-D. If I unplugg and disconnect the device, and plug iback in, everything works again.

I tried RGB-D and PCL demo, and got this issue:

Long story short, it seems like running the examples in this repo have very unexpected results.

I've had to reboot my machine after running some these examples.

I know some others have had similar issues, but there isn't a clear ideas of how to address.

With latest API version (2.7.2.0), the gen2-people-tracker experiment yields inconsistent results, most likely due to video FPS being too high for the nn to produce some valid data which we can process later on - since we only get only 1/2 detections on the video frames while the person is moving, it's almost impossible to detect the person movement direction

This is in reference to experiment - gen2-age-gender
Showing below message when using command

python main.py -vid ./input.mp4
RuntimeError: No available devices

Note; I have not attached any camera device to system

Hi,

Is there any plans to upgrade the people counter example to gen2 api?

Thanks,
Marcelo

Hi, I bought OAK-1.
The depthai_demo.py worked fine then I tried to work depthai-experiments.
I ran people-tracker main.py, but I got the message No USB device [03e7:2485], still looking... 10.045s NOT FOUND, err code 5.
The full log is below.
I'm working on windows 10 and I watched this issue #36
So I used virtualenv and created each environment to depthai-demo.py and people tracker/main.py.
What do I need to do to resolve this?

python .\main.py
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Loading config file
Attempting to open stream config_d2h
watchdog started Successfully opened stream config_d2h with ID #0!

Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: invalid / unprogrammed
D:\home\iprediction\depthAI\depthai-experiments\people-tracker\model\config.json
depthai: Calibration file is not specified, will use default setting;
config_h2d json:
{"_board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":1,"NN_config":{"NN_family":"mobilenet","confidence_threshold":0.5,"output_format":"detection"},"blob0_size":2290560,"blob1_size":0,"calc_dist_to_bb":false,"camera_input":"rgb","cmx_slices":7,"keep_aspect_ratio":true,"num_stages":1,"shaves":7},"app":{"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":69.0,"left_to_rgb_distance_m":0.0,"left_to_right_distance_m":0.03500000014901161,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":69.0,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":false},"camera":{"mono":{"fps":30.0,"resolution_h":720,"resolution_w":1280},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":-1,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->1589
size of json_config_obj that is expected to be sent to config_h2d is ->1048576
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 2290560
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 2290560 bytes to inBlob
!!! XLink write successful: inBlob (2290560)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file D:\home\iprediction\depthAI\depthai-experiments\people-tracker\model\model.blob
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
Input layer :
Name: data
Index: 0
Element type: uint8
Element size: 1byte
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 3, Height : 320, Width : 544]

Output layer :
Name: detection_out
Index: 0
Element type: float16
Element size: 2 bytes
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 1, Height : 200, Width : 7]

CNN to depth bounding-box mapping: start(0, 0), max_size(0, 0)
Host stream start:metaout
Opening stream for read: metaout
Attempting to open stream metaout
Successfully opened stream metaout with ID #3!
Starting thread for stream: metaout
Host stream start:previewout
Opening stream for read: previewout
Attempting to open stream previewout
Started thread for stream: metaout
Successfully opened stream previewout with ID #4!
Starting thread for stream: previewout
depthai: INIT OK!
Started thread for stream: previewout
XLink initialized.
No USB device [03e7:2485], still looking... 10.045s NOT FOUND, err code 5
depthai: Error initializing xlink
device is not initialized
Traceback (most recent call last):
File "", line 1, in
File "C:\Python38\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Python38\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Python38\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Python38\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Python38\lib\runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Python38\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Python38\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "D:\home\iprediction\depthAI\depthai-experiments\people-tracker\main.py", line 13, in
d = DepthAI()
File "D:\home\iprediction\depthAI\depthai-experiments\people-tracker\depthai_utils.py", line 21, in init
raise RuntimeError("Error creating a pipeline!")
RuntimeError: Error creating a pipeline!

Thank u for creating this repository. It really useful
I try to compare the output of mobileface_arcface model in this repository with origin model (mxnet) by cosine distance, the distance is really big (0.2 -> 0.6). Is it a converting model problem? I tried to convert again but problem is not fixed

I was running different scripts to check the if the required library version runs on the system, while I ran across an error in Traingulation 3D Visualiser, so gave a thought of reporting the error faced on "armv7l" machine-system, i.e 32 bit Raspbian OS.
Here's the error -

(venv2) pi@raspberrypi:~/depthai-experiments/triangulation-3D-visualizer $ python3 main.py
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Loading config file
Attempting to open stream config_d2h
watchdog started 
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: valid (v5)
  Board name     : BW1098OBC
  Board rev      : R0M0E0
  HFOV L/R       : 71.86 deg
  HFOV RGB       : 68.7938 deg
  L-R   distance : 7.5 cm
  L-RGB distance : 3.75 cm
  L/R swapped    : yes
  L/R crop region: center
  Rectification Rotation R1 (left):
    0.999987,    0.004518,    0.002240,
   -0.004518,    0.999990,    0.000004,
   -0.002240,   -0.000014,    0.999997,
  Rectification Rotation R2 (right):
    0.999996,   -0.002422,   -0.001414,
    0.002422,    0.999997,   -0.000010,
    0.001414,    0.000007,    0.999999,
  Calibration intrinsic matrix M1 (left):
  857.518066,    0.000000,  637.386780,
    0.000000,  858.455322,  404.491150,
    0.000000,    0.000000,    1.000000,
  Calibration intrinsic matrix M2 (right):
  856.180054,    0.000000,  640.267273,
    0.000000,  856.741455,  398.091125,
    0.000000,    0.000000,    1.000000,
  Calibration rotation matrix R:
    0.999969,    0.006940,    0.003653,
   -0.006941,    0.999976,    0.000005,
   -0.003653,   -0.000031,    0.999993,
  Calibration translation matrix T:
   -7.513971,
    0.018201,
    0.010622,
  Calibration Distortion Coeff d1 (Left):
   -5.192124,   16.073618,    0.000318,   -0.000139,  -16.213690,   -5.244258,   16.288498,
  -16.425783,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
  Calibration Distortion Coeff d2 (Right):
   41.654877, -142.081024,   -0.000621,   -0.001974,  582.817444,   41.766983, -145.366440,
  590.502930,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
/home/pi/depthai-experiments/triangulation-3D-visualizer/models/face-detection-retail-0004/face-detection-retail-0004.json
depthai: Calibration file is not specified, will use default setting;
config_h2d json:
{"_board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":2,"NN_config":{"NN_family":"mobilenet","confidence_threshold":0.5,"output_format":"detection"},"blob0_size":1335872,"blob1_size":532288,"calc_dist_to_bb":false,"camera_input":"left_right","cmx_slices":14,"keep_aspect_ratio":true,"num_stages":2,"shaves":14},"app":{"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":69.0,"left_to_rgb_distance_m":0.0,"left_to_right_distance_m":0.03500000014901161,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":69.0,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":false},"camera":{"mono":{"fps":30.0,"resolution_h":720,"resolution_w":1280},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":-1,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 1335872 bytes to inBlob
!!! XLink write successful: inBlob (1335872)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/pi/depthai-experiments/triangulation-3D-visualizer/models/face-detection-retail-0004/face-detection-retail-0004.blob
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
CNN to depth bounding-box mapping: start(280, 0), max_size(720, 720)
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 532288 bytes to inBlob
!!! XLink write successful: inBlob (532288)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/pi/depthai-experiments/triangulation-3D-visualizer/models/landmarks-regression-retail-0009/landmarks-regression-retail-0009.blob
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
Opening stream for read: metaout
Attempting to open stream metaout
Successfully opened stream metaout with ID #1!
Starting thread for stream: metaout
Opening stream for read: previewout
Attempting to open stream previewout
Started thread for stream: metaout
Successfully opened stream previewout with ID #2!
Starting thread for stream: previewout
Started thread for stream: previewout
Traceback (most recent call last):
  File "main.py", line 51, in <module>
    initialize_OpenGL()
  File "/home/pi/depthai-experiments/triangulation-3D-visualizer/visualizer.py", line 119, in initialize_OpenGL
    gluPerspective(45, (1.0 * display[0] / display[1]), 0.1, 50.0)
  File "/home/pi/venv2/lib/python3.7/site-packages/OpenGL/platform/baseplatform.py", line 425, in __call__
    self.__name__, self.__name__,
OpenGL.error.NullFunctionError: Attempt to call an undefined function gluPerspective, check for bool(gluPerspective) before calling
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
!!! XLink write successful: host_capture (4)
Stopping threads: ...
E: [global] [    851885] [Scheduler00Thr] dispatcherEventSend:53	Write failed (header) (err -1) | event XLINK_READ_REL_REQ

E: [xLink] [    851885] [Scheduler00Thr] sendEvents:1027	Event sending failed
E: [global] [    851885] [python3] addEvent:264	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    851885] [python3] addEventWithPerf:276	 addEvent(event) method call failed with an error: 3
E: [global] [    851885] [python3] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream metaout: ...
Closing stream metaout: DONE.
Thread for metaout finished.
E: [global] [    851886] [Scheduler00Thr] dispatcherEventSend:53	Write failed (header) (err -1) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    851886] [Scheduler00Thr] sendEvents:1027	Event sending failed
Closing stream previewout: ...
E: [global] [    851887] [Scheduler00Thr] dispatcherEventSend:53	Write failed (header) (err -1) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    851887] [Scheduler00Thr] sendEvents:1027	Event sending failed
Closing stream previewout: DONE.
Thread for previewout finished.
Stopping threads: DONE 0.004s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
Closing all observer streams: DONE.
Reseting device: 0.
E: [global] [    851888] [Scheduler00Thr] dispatcherEventSend:53	Write failed (header) (err -1) | event XLINK_RESET_REQ

E: [xLink] [    851888] [Scheduler00Thr] sendEvents:1027	Event sending failed
Reseting: DONE.

The error defined is while using gluPerspective from pyopengl version "3.1.5"

Current implementation of gen2-camera-demo example have a couple of TODO items, some about optimizing the code, some about adding full CLI args support.

These items should be addressed and resolved

Hi,

I'm trying to run a NN model on the RGB preview stream and planned to use the stereo depth to generate the depth of each prediction. Seemed relatively straightforward but I've been encountering a X_Link_Error after adding the StereoDepth pipeline. The RGB pipeline with NN runs perfectly well.

RuntimeError: Couldn't read data from stream: '__rpc_main' (X_LINK_ERROR)

I've taken some steps to troubleshoot the error using the demo scripts. I have isolated it to a difference in depthai versions and am assuming it's compatibility issues between gen1 and gen2 but am uncertain. Do you have any idea what might be going on or how it can be resolved? Wondering if the Module 3 I have is not compatible with gen2's stereodepth?

Some context: I had to flash my device because of some issues and followed the steps in the documentation to setup it up again.

Device: DepthAI: RPi Compute Module 3 Edition

Troubleshooting Findings

1. No errors with depthai.py. The depth value, z is present. This script is automatically started as soon as the device boots/starts.
   Depthai Version Installed: 0.0.10a
   Depthai Dev Version Installed: 8c6328c8acd1086542b47d09b4b192ec1143914c
   

2. Run time error with depthai_demo.py which was launched using test.py
   Depthai Version Installed: 2.5.0.0
   Depthai Dev Version Installed: AttributeError: module 'depthai' has no attribute 'dev_version'
   

3. Ran the gen2 camera demo https://github.com/luxonis/depthai-experiments/blob/f08e06c/gen2-camera-demo/main.py#L49. Both the create_rgb_pipeline() and create_mono_pipeline() pipelines ran successfully but encountered the same X_Link_Error runtime error in the create_stereo_depth_pipeline()
   

[Error] Reported while running Gen2 OCR example on armv7l;
Using libraries -
numpy==1.19.5
opencv-python==4.5.1.48
depthai==0.0.2.1+e88c98c8a7681dd16cd754a832df1165380ac305

Error -

(venv2) pi@raspberrypi:~/depthai-experiments/gen2-ocr $ python3 main.py 
main.py:15: DeprecationWarning: setCamId() is deprecated, use setBoardSocket() instead.
  colorCam.setCamId(0)
terminate called without an active exception
Aborted

My depthai version is 2.0.0.1

When I try to run the collecting data code, I find there are some AttributeError.
For example, 'depthai.Device' object has no attribute 'create_pipeline'

Are the codes in depthai-experiments incompatible with a new version of depthai?
Or is there a plan to reimplement them using a new version depthai?
Thanks.

We should migrate rstp streaming example to Gen2.

Together with this migration, we can try to follow this suggestion in order to optimize CPU usage on the host

Demo-UI (here)
Application used to demonstrate various capabilities of the DepthAI platform. Contains examples with descriptions, console outputs and preview windows.

This part in readme contains a link below. It redirects to 404, so needs a fix or removal

I converted using the blob converter and choose the right version 2020.1 but I still get an error, so how can I reshape my inputs from 544x320->672x383?

Prepare an example with object detection, that will pull the closest depth point from the object and return it as the object distance.

Related to discussion on discuss - https://discuss.luxonis.com/d/67-distance-to-closest-point-on-an-object-bw1098obc

This demo could use the ROI crop capability of Gen2 to pull out only the depth map for detected objects, and then pass this (much smaller set of) data back to the host where a variety of selection mechanisms can be used:

1. Front of the object
2. Average of the object (but not the background)
3. A given edge of the object
4. Some subset of the middle of the object.
5. The front say 20% of the object.

After the depthai lib update, the Demo works well, however the text recognition part after the detection inferences a bit too fast for the human eye to spot.
Perhaps a possible fix could be letting the text spotting model make inferences, but, the text recognition model output the recognised text on the different frame after a specific duration.
(last_text_recognised.time - current_text_recognised.time) = 1second

For every example in this repo, I get the following warning (or something similar):

[14442C10A1E2F0D000] [22.544] [NeuralNetwork(2)] [warning] Network compiled for 13 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance

I've looked but I find nothing about how to recompile for the '6 shaves' suggested.
Maybe I'm overthinking it?

A Google search isn't helpful ...unless you want to learn about facial hair.

Many thanks in advance!

I tried to migrate this example to latest DepthAI Gen2 API (commit hash 45765d01ad9c7d8d9ae99ca0f76c5b03b706a1c4), and got the following error after running against this version

Traceback (most recent call last):
  File "/home/vandavv/dev/luxonis/depthai-experiments/gen2-ocr/main.py", line 12, in <module>
    colorCam.setAutoFocusMode(depthai.RawCameraControl.AutoFocusMode.AUTO)
AttributeError: 'depthai.ColorCamera' object has no attribute 'setAutoFocusMode'

python main.py
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Loading config file
Attempting to open stream config_d2h
watchdog started
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: valid (v3)
Board name : BW1098OBC
Board rev : R0M0E0
HFOV L/R : 71.86 deg
HFOV RGB : 68.7938 deg
L-R distance : 7.5 cm
L-RGB distance : 3.75 cm
L/R swapped : yes
L/R crop region: center
Calibration homography:
1.011238, -0.021879, -10.484687,
0.026843, 1.010301, -22.506275,
0.000004, 0.000005, 1.000000,
CNN configurations read: /home/iotuser/oakd/depthai-experiments/collecting-training-data/mobilenet-ssd/mobilenet-ssd.json
depthai: Calibration file is not specified, will use default setting;
config_h2d json:
{"_board":{"_homography_right_to_left":[0.988068163394928,0.0029474012553691864,5.067617416381836,-0.008765067905187607,0.9921473264694214,-8.795275688171387,-8.449587767245248e-06,-3.603489403758431e-06,1.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"left"},{"name":"right"},{"name":"previewout"},{"name":"disparity"}]},"ai":{"NCEs":1,"blob0_size":14489664,"blob1_size":0,"calc_dist_to_bb":false,"camera_input":"rgb","cmx_slices":7,"keep_aspect_ratio":true,"num_stages":1,"shaves":7},"app":{"sync_video_meta_streams":false},"board":{"clear-eeprom":false,"left_fov_deg":69.0,"left_to_rgb_distance_m":0.0,"left_to_right_distance_m":0.03500000014901161,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":69.0,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":false},"camera":{"mono":{"fps":30.0,"resolution_h":720,"resolution_w":1280},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"confidence_threshold":0.5,"depth_limit_mm":10000,"padding_factor":0.30000001192092896},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 5000 bytes to config_h2d
!!! XLink write successful: config_h2d (5000)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 14489664
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 14489664 bytes to inBlob
!!! XLink write successful: inBlob (14489664)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/iotuser/oakd/depthai-experiments/collecting-training-data/mobilenet-ssd/mobilenet-ssd.blob
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
CNN input width: 300
CNN input height: 300
CNN input num channels: 3
CNN to depth bounding-box mapping: start(318, 38), max_size(643, 643)
Host stream start:left
Opening stream for read: left
Attempting to open stream left
Successfully opened stream left with ID #1!
Starting thread for stream: left
Host stream start:right
Opening stream for read: right
Attempting to open stream right
Started thread for stream: left
Successfully opened stream right with ID #2!
Starting thread for stream: right
Started thread for stream: right
Host stream start:previewout
Opening stream for read: previewout
Attempting to open stream previewout
Successfully opened stream previewout with ID #3!
Starting thread for stream: previewout
Started thread for stream: previewout
Opening stream for read: disparity
Attempting to open stream disparity
Successfully opened stream disparity with ID #4!
Starting thread for stream: disparity
Started thread for stream: disparity
depthai: INIT OK!
previewout
35.058774208 1 previewout
previewout
35.092103955 2 previewout
previewout
35.12543381 3 previewout
previewout
35.158761742 4 previewout
previewout
35.192089875 5 previewout
previewout
35.225418364 6 previewout
previewout
35.258747274 7 previewout
previewout
35.292075922 8 previewout
previewout
35.325404692 9 previewout
previewout
35.358733174 10 previewout
previewout
35.392061451000004 11 previewout
previewout
35.425391135 12 previewout
previewout
35.458720438 13 previewout
previewout
35.492051115 14 previewout
previewout
35.525376275 15 previewout
previewout
35.558704762 16 previewout
previewout
35.592033577 17 previewout
previewout
35.625362382 18 previewout
previewout
35.658689817 19 previewout
previewout
35.692018335 20 previewout
previewout
35.725347987 21 previewout
previewout
35.758676428 22 previewout
previewout
35.79200528 23 previewout
previewout
35.825334102 24 previewout
previewout
35.858662675 25 previewout
previewout
35.891991267 26 previewout
previewout
35.925320787 27 previewout
previewout
35.958651684 28 previewout
previewout
35.99197622 29 previewout
previewout
36.0253047 30 previewout
previewout
36.058634515 31 previewout
previewout
36.091967362 32 previewout
previewout
36.125291681 33 previewout
previewout
36.158626598 34 previewout
previewout
36.191948952 35 previewout
previewout
36.225277564 36 previewout
previewout
36.258606224 37 previewout
previewout
36.291935172 38 previewout
previewout
36.325262637 39 previewout
previewout
36.358591198 40 previewout
previewout
36.391920652 41 previewout
previewout
36.425251405 42 previewout
previewout
36.458578498 43 previewout
previewout
36.491906894 44 previewout
previewout
36.525235354 45 previewout
previewout
36.558563964 46 previewout
previewout
36.591892842 47 previewout
previewout
36.625222194 48 previewout
previewout
36.658549788 49 previewout
previewout
36.6918775 50 previewout
previewout
36.725207052 51 previewout
previewout
36.758536677 52 previewout
previewout
36.791864477 53 previewout
previewout
36.825193174 54 previewout
previewout
36.858523621 55 previewout
previewout
36.8918504 56 previewout
previewout
36.925182752 57 previewout
previewout
36.958507818 58 previewout
previewout
36.991835745 59 previewout
previewout
37.02516402 60 previewout
previewout
37.05849328 61 previewout
previewout
37.091822035 62 previewout
previewout
37.125151017 63 previewout
previewout
37.158480012 64 previewout
previewout
37.191808152 65 previewout
previewout
37.225136797 66 previewout
previewout
37.258466784 67 previewout
previewout
37.291795132 68 previewout
previewout
37.325121918 69 previewout
previewout
37.358451511 70 previewout
previewout
37.39178542 71 previewout
previewout
37.425109257 72 previewout
previewout
37.45843728 73 previewout
previewout
37.491765974 74 previewout
previewout
37.525094554 75 previewout
previewout
37.558423462 76 previewout
previewout
37.59175412 77 previewout
previewout
37.625080721 78 previewout
previewout
37.658408318 79 previewout
previewout
37.691736797 80 previewout
previewout
37.72506608 81 previewout
previewout
37.758394715 82 previewout
previewout
37.791728524 83 previewout
previewout
37.825066584 84 previewout
previewout
37.858380832 85 previewout
previewout
37.891709482 86 previewout
previewout
37.925038291999996 87 previewout
previewout
37.958366975 88 previewout
previewout
37.991694798 89 previewout
previewout
38.025024235 90 previewout
previewout
38.05835226 91 previewout
previewout
38.091680197 92 previewout
left
38.11159051 1 left
right
38.111600905 1 right
previewout
38.125014062 93 previewout
disparity_color
Traceback (most recent call last):
File "main.py", line 69, in
print(packet.getMetadata().getTimestamp(), packet.getMetadata().getSequenceNum(), packet.stream_name)
AttributeError: 'NoneType' object has no attribute 'getTimestamp'
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
!!! XLink write successful: host_capture (4)
Stopping threads: ...
Closing stream right: ...
E: [global] [ 715628] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_READ_REL_REQ

E: [xLink] [ 715628] [Scheduler00Thr] sendEvents:1027 Event sending failed
E: [global] [ 715628] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [ 715628] [Scheduler00Thr] sendEvents:1027 Event sending failedE: [global] [ 715628] [python] addEvent:262
E: [global] [ 715628] [Scheduler00Thr] dispatcherEventSend:53 Condition failed: event->header.flags.bitField.ack != 1Write failed (header) (err -4) | event XLINK_READ_REL_REQ

Closing stream right: DONE.
Thread for right finished.
E: [global] [ 715628] [python] addEventWithPerf:276 addEvent(event) method call failed with an error: 3
E: [global] [ 715628] [python] XLinkReadData:156 Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream left: ...
E: [xLink] [ 715628] [Scheduler00Thr] sendEvents:1027 Event sending failed
E: [global] [ 715628] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [ 715628] [Scheduler00Thr] sendEvents:1027 Event sending failedClosing stream left: DONE.
Thread for left finished.

Closing stream previewout: ...
E: [global] [ 715629] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

Closing stream previewout: DONE.
Thread for previewout finished.
E: [xLink] [ 715629] [Scheduler00Thr] sendEvents:1027 Event sending failed
Closing stream disparity: ...
E: [global] [ 715629] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [ 715629] [Scheduler00Thr] sendEvents:1027 Closing stream disparity: DONE.
Thread for disparity finished.
Event sending failed
Stopping threads: DONE 0.758s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
Closing all observer streams: DONE.
Reseting device: 0.
E: [global] [ 715629] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [ 715629] [Scheduler00Thr] sendEvents:1027 Event sending failed
Reseting: DONE.

The gen2-mjpeg-streaming example doesn't work on my Windows machine. If I use the library version specified inside the requirements.txt, I get the following error:

Traceback (most recent call last):
  File "main.py", line 75, in <module>
    pipeline = dai.Pipeline()
TypeError: depthai.Pipeline: No constructor defined!

The specified depthai version inside the requirements.txt is 0.3.0.0+aeda4a9fdef6edc9f826b7dc354a123d1611a7c6 but as far as I know this isn't gen 2 yet so I tried to update the depthai version to 2.4.0.0. After that, the TCP server works fine, but the HTTP Server crashes when I visit the page (localhost:8090).

If I remove line 47, the server stops crashing, but no image is displayed. Any help is highly appreciated.

In Gen2 depthai_demo.py there is a disparity confidence slider (scales from 0 to 255). It allows one to "tune-in" between false-positive and false-negatives. Examples below for my ceiling in the background:

This would be very interesting to play with in the WLS example:

As you can see that my ceiling isn't quite captured with the standard disparity confidence threshold, but from the depthai_demo.py Gen1 example the ceiling actually can be mostly/accurately estimated.

And likely the disparity confidence being a bit too low for this environment is a bit of the cause of the flashiness of my ceiling shown below:
https://youtu.be/IEqg0R2xA3g

depthai-experiments/gen2-tf-image-classification/

I just tried to run the code of converting OpenVINO IR to Blob format in the colab file.
!/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/myriad_compile -m /content/flower/IR_models/FP32/flower.xml -ip U8 -VPU_NUMBER_OF_SHAVES 4 -VPU_NUMBER_OF_CMX_SLICES 4

But, it produces this:
/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/myriad_compile: error while loading shared libraries: libinference_engine.so: cannot open shared object file: No such file or directory.

Could you help see why this is happening?

Is there a reason why I'm getting a 5second delay/latency on the video feed with that demo? Is that a Mac OSX specific issue or it's the pipeline (got too deep?). Thank you.

Start with the why:

Now that we have the general text-detection + optical character recognition pipeline working (here), it would great to show a specific example of instead of doing overall text detection, to only detect license plates and then do OCR on them instead.

So below shows running the general OCR example on license plates:

With a license-plate detector as the first stage (instead of a general text detector), likely things like the DVD 493 plate would be correctly detected as one string instead of 2, for example. And other attributes (like country, state, etc.) could be pulled off, as in our manual/hard-coded example of such a pipeline, here.

Move to the how:

Use the vehicle-license-plate-detection-barrier-0106 detector instead of the EAST general text detector as the first-stage detector (and likely return additional metadata it may provide) and feed this into the OCR.

An example of running this first-stage model is below:

And then in addition to this model as the first-stage, we should likely use vehicle-attributes-recognition-barrier-0039 to get vehicle attributes and license-plate-recognition-barrier-0001 to actually do the OCR of the plate (as in the hard-coded example, here).

Move to the what:

Make a Gen2 pipeline example/reference application specific for license plate detection and optical character recognition (OCR).

Hi,

With depthai version "0.2.0.1+f91a36fac86a0f4383e863e04cb651869a1603ac" I'm getting device is not initialized error on raspberry pi. Atteched is the error log.

XLink initialized.
Sending internal device firmware
Failed to boot the device: 1.2-ma2480, err code 3
depthai: Error initializing xlink
2020-11-24 16:54:34,380 - depthai_utils - INFO - Creating DepthAI pipeline...
device is not initialized

With the latest API version (2.7.2.0), the gen2-gaze-estimation experiment yields an incorrect gaze vector (random gaze direction).

Worth checking how can we use the new detection network nodes to optimize this experiment (or maybe even fix this way)

With the TensorFlow Image Classification example added, we noticed that even though the example works correctly and outputs results as in TF reference, the detected flower is not correct.

We should take a deeper look at the conversion steps in this notebook, to verify if they are the issue here

Hello,

I have been trying the license plate recognition sample and it is not working as shown in the video. I used the chinese_traffic.mp4 as input and I only get the rgb and attributes window. The license plates are not being recognized. Is anything missing to enable the plate recognition?

Thanks,
Marcelo

This example shows how to use WLS filtering on Gen1. Now that Gen2 is going mainstream we should update it to use Gen2.

https://github.com/luxonis/depthai-experiments#host-side-wls-filter-here

This might have been referenced too many times, but I'm opening this over here for reference.
The gen2-camera-demo starts for a few seconds, and closes immidiately triggering an error -

StereoDepth config options:
    Left-Right check:   True
    Extended disparity: False
    Subpixel:           True
    Median filtering:   MedianFilter.MEDIAN_OFF
Creating Stereo Depth pipeline: MONO CAMS -> STEREO -> XLINK OUT
Creating DepthAI device
Starting pipeline
Traceback (most recent call last):
  File "main.py", line 306, in <module>
    test_pipeline()
  File "main.py", line 296, in test_pipeline
    image = q.get()
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'depth' (X_LINK_ERROR)'

Sometimes with the error -

StereoDepth config options:
    Left-Right check:   True
    Extended disparity: False
    Subpixel:           True
    Median filtering:   MedianFilter.MEDIAN_OFF
Creating Stereo Depth pipeline: MONO CAMS -> STEREO -> XLINK OUT
Creating DepthAI device
Starting pipeline
Segmentation fault

It is using updated lib - depthai==2.0.0.1, and running the stereo demo, not point cloud, since it's running on armv7l - Rpi 4b 8gb

I checked the CPU load, and it was barely any, but still reciprocating an error.
Have kept this open for reference.

hi ,
the gaze net output pitch and yaw angles. I think it's in head coordinate system, then, how can I convert it to the camera coordinate system?

I installed the requirements and plugged the OAK-D depthAI camera.
When running main.py with parameter -cam, I get the message below:

Traceback (most recent call last):
  File "main.py", line 327, in <module>
    Main(camera=args.camera).run()
  File "main.py", line 104, in __init__
    self.start_pipeline()
  File "main.py", line 176, in start_pipeline
    self.device = depthai.Device()
RuntimeError: Failed to find device after booting, error message: X_LINK_DEVICE_NOT_FOUND

The depthAi_demo script from Luxonis works fine with the camera.

[Update]
In fact, the depathAi demo worked fine before I installed the requirements for the gaze detection.
Now, it was broken due to a version conflict and I re-installed the depthAI requirements for the demo.

Now the demo works fine but I get a different error for the gaze detection:

Traceback (most recent call last):
  File "main.py", line 327, in <module>
    Main(camera=args.camera).run()
  File "main.py", line 103, in __init__
    self.create_pipeline()
  File "main.py", line 108, in create_pipeline
    self.pipeline = depthai.Pipeline()
TypeError: depthai.Pipeline: No constructor defined!

What do I need to do to resolve this?

Hello,

I am running the gen2 palm detection sample and getting the following error. Anything that needs to be done to fix this issue? I am attaching the error in the console.

python3 main.py --cam
Loading pipeline...
Creating pipeline...
Creating Color Camera...
Creating models/palm_detection_openvino_2021.3_6shave.blob Neural Network...
linked cam.preview to model_nn.input
Pipeline created.
Starting pipeline...
[14442C100126C6D200] [372.086] [NeuralNetwork(2)] [error] Neural network blob compiled with uncompatible openvino version. Selected openvino version 2021.2. If you want to select an explicit openvino version use: setOpenVINOVersion while creating pipeline.
[14442C100126C6D200] [372.088] [NeuralNetwork(2)] [error] Neural network blob compiled with uncompatible openvino version. Selected openvino version 2021.2. If you want to select an explicit openvino version use: setOpenVINOVersion while creating pipeline.
^CTraceback (most recent call last):
File "main.py", line 468, in
Main(camera=args.camera).run()
File "main.py", line 252, in run
self.run_camera()
File "main.py", line 226, in run_camera
in_rgb = self.preview.tryGet()

Thanks,
Marcelo

The module https://github.com/luxonis/depthai-experiments/tree/master/two-stage-inference has been using the depthai version depthai==0.3.0.0+aeda4a9fdef6edc9f826b7dc354a123d1611a7c6

This can be changed to depthai==2.0.0.1, to make it uniform with other modules.

Future update - Can be integrated to use facial-landmarks-35-adas-0002.blob instead of landmarks-regression-retail-0009

Ability to create a room map based on the depth readings, similarly to the gif below

Inspiration: https://github.com/xdspacelab/openvslam

Throws the following error:

Traceback (most recent call last):
  File "main.py", line 10, in <module>
    from tracker import Tracker
  File "/home/user/collision-avoidance/tracker.py", line 4, in <module>
    from scipy.spatial import distance as dist
ModuleNotFoundError: No module named 'scipy'

Idea is to use DepthAI to recognize chess pieces on the chessboard, like in the gif below

Dataset and useful links can be found here - https://public.roboflow.com/object-detection/chess-full