This repository is a prototype application of SideGuide, a publicly available benchmark dataset for sidewalk navigation.
SideGuide is a publicly available benchmark designed to assist individuals with sidewalk navigation chellenges.
This repository contains the PyTorch code for the "Pyramid Stereo Matching Network" paper (CVPR 2018) by Jia-Ren Chang and Yong-Sheng Chen, as well as the code for the "YOLOv3: An Incremental Improvement" paper by Joseph Redmon and Ali Farhadi. More details: http://pjreddie.com/darknet/yolo/

In ROS, the following connections exist between nodes:

• The Darknet_ros (YOLO) node subscribes to the left image from the ZED camera and publishes information about bounding boxes for detected objects.
• The PSMnet node subscribes to the left and right (stereo) image topics from the ZED camera, estimates disparity, and obtains depth information through the network.
• Finally, by combining the information obtained from the object detection and depth estimation networks, the system displays information for each recognized object, including the type of object and the distance in meters from the camera to the detected object.
  

• To obtain the model trained with the SideGuide Object Detection / Instance Segmentation dataset and find the final AP-50 performance, please refer to the "Download" section.

• For the final performance of the model trained with the Stereo Matching (disparity estimation) dataset and to download the trained model, please refer to the "Results on NIA Sidewalk Dataset" section.

인도보행영상 객체인식/개별 객체분할 데이터셋으로 학습한 모델의 최종 AP-50 성능과 학습된 모델 다운로드 링크는 Download을 참조하세요. 거리(깊이) 추정 데이터셋으로 학습한 모델의 최종 성능과 학습된 모델 다운로드 링크는 Results on NIA Sidewalk Dataset을 참조하세요.

• The sidewalk dataset (SideGuide): [Download Link] for Korean access

⭐⭐ 🔥 NEW 🔥 ⭐⭐

• Please fill out [this form] to download SideGuide, and we will send you the download link shortly after approval.
  

  Will be updated soon!
  We are currently in the process of creating a new landing page to make the dataset available for download without any national restrictions. Once it's ready, we will share the page link here as well. We also aim to have it prepared soon. Thank you for your patience. (10/2023)

Our current implementation is tested on:

• ZED_SDK v2.8.3
• ROS Kinetic
• openCV 2.4.9
• Python 2.7
• PyTorch 1.0.0
• torchvision 0.2.0

Download the two pre-trained weights from our sidewalk dataset--SideGuide.

• YOLOv3 trained on the SideGuide can be downloaded from here and save it to catkin_workspace/src/darknet_ros/darknet_ros/yolo_network_config/weights .

• PSMNet trained on the SideGuide can be downloaded from here and save it to catkin_workspace/src/psmnet/src/trained .

cd catkin_workspace/src
git clone https://github.com/ChelseaGH/sidewalk_prototype_AI_Hub.git
cd ../
catkin_make    

Open a new terminal

$ rosalauch zed_wrapper zed.launch

➜ ZED CAMERA Connected.

Open a new terminal

$ roslaunch darknet_ros yolo_v3_sidewalk.launch

➜ Real-time detection results(Bounding Boxes) are shown on the zed camera image.

Open a new terminal

$ rosrun psmnet psm_re.py

➜ Detected objects and estimated depths are shown in the terminal window.

• Ubuntu 16.04.5 LTS (64-bit)
• CUDA Version: 9.0.176

[email protected]

SideGuide Benchmark:

Copyright (c) 2021 AI Hub

All rights reserved.

The copyright notice must be included when using the data and also for secondary works utilizing this data. This data is built to develop AI technology such as intelligent products, services and can be used for commercial or non-commercial purposes for research and development in various fields.

@inproceedings{park2020sideguide,
  title={SideGuide: A Large-scale Sidewalk Dataset for Guiding Impaired People},
  author={Park, Kibaek and Oh, Youngtaek and Ham, Soomin and Joo, Kyungdon and Kim, Hyokyoung and Kum, Hyoyoung and Kweon, In So},
  booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={10022--10029},
  year={2020},
  organization={IEEE}
}

M. Bjelonic "YOLO ROS: Real-Time Object Detection for ROS", URL: https://github.com/leggedrobotics/darknet_ros, 2018.

@misc{bjelonicYolo2018,
  author = {Marko Bjelonic},
  title = {{YOLO ROS}: Real-Time Object Detection for {ROS}},
  howpublished = {\url{https://github.com/leggedrobotics/darknet_ros}},
  year = {2016--2018},
}

@inproceedings{chang2018pyramid,
  title={Pyramid Stereo Matching Network},
  author={Chang, Jia-Ren and Chen, Yong-Sheng},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={5410--5418},
  year={2018}
}