Network Video Recorder
Details | |
---|---|
Target OS: | Ubuntu* 18.04 LTS |
Programming Language: | Python 3.5 |
Time to Complete: | 45 min |
What It Does
The need to accommodate high resolution video for management, storage, and viewing are essential for smart video surveillance systems ensuring the highest standard of security. This solution demonstrates how to implement and take advantage of Intel hardware platforms for an end to end video analytics involving decoding, encoding, and optimization using various media stacks.
What You Will Learn
This application uses Intel® Distribution of OpenVINO™ toolkit for encoding and decoding high resolution video streams into a server machine for storage and management.
Gain some insights into the following solutions:
• Create and run inference workloads that provide low latency video processing
• Encode and Decode Multiple Video Stream
• Gather data and send back to the edge and or datacenter
Learn to build and run an application with these capabilities:
- Decode and Decode using a multimedia framework library
- Transform and resize frames using computer vision
- Enable Deep Learning to classify frames
- Run filters and count matches in the post-process
- Create a detector processor
- Optimize solution on various Intel hardware
Requirements
Hardware
- UP2* AI Vision Developer Kit + USB webcam
Software
-
Ubuntu* 18.04 LTS
Note: We recommend using a 4.14+ Linux* kernel with this software. Run the following command to determine the kernel version:uname -a
-
Imutils
-
OpenCL™ Runtime Package
-
Intel® Distribution of OpenVINO™ toolkit 2020 R3 Release
-
A multipurpose computer vision solution.
-
GStreamer
How It Works
This application uses 2 video producing terminals in which their streams are decoded into a server machine.
-
The video streams are captured using the multimedia frameworks and encoded into a format that can be accommodated by the server machine.
-
The stored video in the server machine is then streamed into a computer vision application which performs inference detecting people.
-
This inferred video is then sent to an additional system for further analysis.
Setup
Get the code
Clone the reference implementation
sudo apt-get update && sudo apt-get install git
git clone https://github.com/intel-iot-devkit/network-video-recorder.git
Install OpenVINO
Refer to Install Intel® Distribution of OpenVINO™ toolkit for Linux* to learn how to install and configure the toolkit.
Install the OpenCL™ Runtime Package to run inference on the GPU, as shown in the instructions below. It is not mandatory for CPU inference.
Which model to use
This application uses the person-detection-retail-0013 Intel® model, that can be downloaded using the model downloader. The model downloader downloads the .xml and .bin files that will be used by the application.
To download the models and install the dependencies of the application, run the below command in the network-video-recorder
directory:
./setup.sh
Setup the Environment
Configure the environment to use the Intel® Distribution of OpenVINO™ toolkit by exporting environment variables:
source /opt/intel/openvino/bin/setupvars.sh
Note: This command needs to be executed only once in the terminal where the application will be executed. If the terminal is closed, the command needs to be executed again
Running The Application
Though the application uses Gstreamer for encoding and decoding high resolution video streams from multiple devices into a server machine for storage and management, it can also use input videos and also camera feed from the host device for the same task.
Which Input Video to use
The application works with any input video. Sample videos are provided here.
For first-use, we recommend using the face-demographics walking.mp4 and people-detection.mp4 video which will be present in the resources/
directory after the execution of setup.sh
script.
Running on the CPU
A user can specify a target device to run on by using the device command-line argument -d
followed by one of the values CPU
, GPU
,MYRIAD
or HDDL
.
Though by default application runs on CPU, this can also be explicitly specified by -d CPU
command-line argument:
python3 network_video_recorder.py -d CPU -i ./resources/face-demographics-walking.mp4 -i2 ./resources/people-detection.mp4 -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml
To run the application on sync mode, use -f sync
as command line argument. By default, the application runs on async mode.
To run with multiple devices use -d MULTI:device1,device2. For example: -d MULTI:CPU,GPU,MYRIAD
Running on the GPU
- To run on the integrated Intel® GPU with floating point precision 32 (FP32), use the
-d GPU
command-line argument:
python3 network_video_recorder.py -d GPU -i ./resources/face-demographics-walking.mp4 -i2 ./resources/people-detection.mp4 -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml
FP32: FP32 is single-precision floating-point arithmetic uses 32 bits to represent numbers. 8 bits for the magnitude and 23 bits for the precision. For more information, click here
- To run on the integrated Intel® GPU with floating point precision 16 (FP16):
python3 network_video_recorder.py -d GPU -i ./resources/face-demographics-walking.mp4 -i2 ./resources/people-detection.mp4 -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml
FP16: FP16 is half-precision floating-point arithmetic uses 16 bits. 5 bits for the magnitude and 10 bits for the precision. For more information, click here
Note: With GPU, there is a known issue of improper detections where the detections for one of the input videos overlaps with the other one.
Running on the Intel® Neural Compute Stick
To run on the Intel® Neural Compute Stick, use the -d MYRIAD
command-line argument:
python3 network_video_recorder.py -d MYRIAD -i ./resources/face-demographics-walking.mp4 -i2 ./resources/people-detection.mp4 -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml
Running on the Intel® Movidius™ VPU
To run on the Intel® Movidius™ VPU, use the -d HDDL
command-line argument:
python3 network_video_recorder.py -d HDDL -i ./resources/face-demographics-walking.mp4 -i2 ./resources/people-detection.mp4 -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml
Note: The Intel® Movidius™ VPU can only run FP16 models. The model that is passed to the application, through the -m <path_to_model>
command-line argument, must be of data type FP16.
If the user wants to use any other videos, they can be used by providing the associated path in the above command.
Using the Camera Stream instead of video
The command to be executed when we want to use the camera stream is a s follows:
python3 network_video_recorder.py -i cam -i2 cam -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml
In this case also, the user can specify a target device to run on by using the device command-line argument -d
followed by one of the values CPU
, GPU
,MYRIAD
or HDDL
.
Using Gstreamer
Follow the following steps to display person detection results on a host machine from separate indepenedent individual system feeds.
- Stream camera with Gstreamer on Up-Squared board:
$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! "image/jpeg,width=640, height=480,framerate=30/1" ! rtpjpegpay ! udpsink host=<host_ip> port=5000
- Stream camera with Gstreamer on Up-Squared board 2:
gst-launch-1.0 -v v4l2src device=/dev/video0 ! "image/jpeg,width=640, height=480,framerate=30/1" ! rtpjpegpay ! udpsink host=<host_ip> port=5001
- Run detection script on host machine:
python3 network_video_recorder.py -i gstreamer -i2 gstreamer -m /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml
Similar to the input video and camera scenarios, the detction script can be run on host machine using any of the target device that the user desires by using the device command-line argument -d
followed by one of the values CPU
, GPU
,MYRIAD
or HDDL
.