A low-cost People Flow Analysis System developed using NVIDIA's Jetson and Deepstream SDK, integrated with high quality open source software and library.

The system is able to monitor every person within the camera vision. Each person detected will be tracked giving a unique track ID and a green bounding box will be drawn on it. When the detected person stay on the same spot for a certain duration, the system will send a message alert to an authorized Azure Iot Hub and android mobile phone. The message alert contains time, track id and location.

There is also a calibration process to define the ground surface mapping from the camera view. The calibration input will be set by the user.

• NVIDIA Jetson Platform
• JetPack 4.6
• USB webcam or 8 MP Raspberry Pi Camera or Video(.h264 format)

Option:

• 7" 1024x600 capacitive touch monitor
• Android Phone

Test on:

• Jetson AGX Xavier, NX, JetPack 4.6, Video, USB webcam.

• NVIDIA DeepStream 6.0
• MQTT Client
• Mosquitto MQTT Broker
• Download sample data
• Download trained model data

Option:

• Azure IoT Device
• Open Distro for ElasticSearch
• Open Distro for Kibana
• Azure Virtual Machine

$ git clone <this repo>

Installing the five methods. Using the DeepStream tar package here.
Download package here .

$ sudo tar -xvf deepstream_sdk_<deepstream_version>_jetson.tbz2 -C /
$ cd /opt/nvidia/deepstream/deepstream-6.0
$ sudo ./install.sh
$ sudo ldconfig

See NVIDIA DeepStream SDK Developer Guide , and more.

See NVIDIA-AI-IOT/deepstream_python_apps/bindings/README.md.

$ cd /opt/nvidia/deepstream/deepstream-6.0/source/
$ sudo git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps.git
$ cd deepstream_python_apps/bindings

$ sudo pip3 install pybind11
$ sudo git submodule update --init
$ sudo mkdir build && cd build
$ sudo cmake ..  -DPYTHON_MAJOR_VERSION=3 -DPYTHON_MINOR_VERSION=6 -DPIP_PLATFORM=linux_aarch64 -DDS_PATH=/opt/nvidia/deepstream/deepstream-6.0/
$ sudo make

Please download the sample data here. (Google Drive link)

Download the models dir from here. (Google Drive link)

Tree after download models dir:

<this repo>
	|- common
	|- config
	|- data
	|- kibana
	|- models
		|- peoplenet (in 4 files)
	|- mobile
	|- scripts
	|- src

$ pip3 install paho-mqtt

$ sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
$ sudo apt update
$ sudo apt install mosquitto

Please access the APK link below in your Android phone and allow to install.

PFAClientApp-Alpha2.apk

1. On Linux terminal, start the mosquitto message broker service

   $ sudo service mosquitto start
   

2. Using "PFA Client" mobile application, select to connection button. Once connection is established, the application is now readdy to receive the message alerts.

3. Edit the MQTT Host in config/pfa_config.ini (which should be line 34)

   #Your Jetson IP address 
   HOST ="<Jetson IP address>"
   

Note:

1. Stopping the mosquitto message broker service will disable mobile connections to all PFA client applications

   $ sudo service mosquitto stop
   

2. Mobile and Jetson must be connected in the same area network.

$ pip3 install azure-iot-device

Please refer to the README guide in Kibana dir for the setup details.

$ sudo service mosquitto start

$ cd script
$ python3 main.py [VIDEO/FILE/PATH]

#Use sample data
$ python3 main.py ../data/2020-08-27_15-12-31.264

1. Edit the Camera flag in config/pfa_config.ini (witch should be line 97):

#USB webcom (default)
COM_MODE = 0

#Raspberry Pi Camera
COM_MODE = 1

2. Run the command

$ cd script
$ python3 main.py

After run the command, wait for stream to show.

1. While streaming, double click 4 corners(top-left, top-right, bottom-left, bottom-right)

2. Then, stream with 2D map view will appear

3. People detection and tracking will be automatically displayed after the calibration process

Note: Edit the config/pfa_config.ini of initial configuration setting as you wish. The settings for the following are included in the file:

• IOT Hub Client Setting
• Mobile Client Setting
• Aspect Ratio
• Pixel dimension per feet (Grid Size)
• Dwell Time limitation (in second) for alert
• Dwell Time Average limitation for alert

Data analysis and visualization with Open Distro for Kibana.
After access and login the Open Distro for Kibana, create Visualization and Dashboard. Refer to the README guide in kibana directory for the creating visualization details.

Copyright (C) 2020, Ryoyo-NV All rights reserved.
The models and sample data are available for research, development, and PoC purposes only.