Deep learning based object tracking with line crossing and area intrusion detection

License: Apache License 2.0

Python 100.00%

intel openvino openvino-toolkit inference object-detection object-tracking pedestrian-detection pedestrian-tracking line-crossing-detection intrusion-detection

object-tracking-line-crossing-area-intrusion's Introduction

Object Tracking with Line Crossing and Area Intrusion Detection

This program uses an object detection deep learning model and a re-identification model to find and track the objects in a movie. Then, the program will track the trajectory of the objects and check if the objects cross the defined virtual lines or the objects are inside the defined areas. The program uses Intel(r) Distribution of OpenVINO(tm) toolkit for deep learning inferencing.
To track the objects, the program extracts the feature vectors from the objects which consists of 256 FP values with the re-identification DL model. The program can identify which object is most similar to the objects in the previous frame and record the trajectory. The L2 norm (or the distance) of those feature vectores represents the similarity of the objects. This program uses cosine distance to check the similarity between the objects.
The line crossing check and area intrusion check are done by regular (meaning, non deep learning based) arithmetic algorithm.
The latest version playbacks sound when an event such as crossing a line or entering to an area happens.
You can learn how to use line-crossing and area intrusion function from simple test program (test-line-cross-area-intrusion.py)

このプログラムはディープラーニングの物体検出モデルと特徴抽出(re-identification)モデルを用いて、ムービーフレーム中のオブジェクトを検出し、追跡するプログラムです。プログラムは見つけたオブジェクトの軌跡を記録し、オブジェクトが定義された仮想ラインを超えたり定義されたエリアに侵入したことを検出します。ディープラーニングの推論にはIntel(r) Distribution of OpenVINO(tm) toolkitを利用しています。
オブジェクトの追跡のために、プログラムは特徴抽出モデルを使用して見つけたオブジェクトから256個の浮動小数点数値からなる特徴ベクトルを抽出します。この特徴ベクトル同士のL2ノルム(=距離)がオブジェクト同士の類似度を表します。このプログラムではノルムの計算にコサイン距離を使用しています。
ラインクロスとエリア侵入の判定には通常の(ディープラーニングベースではない)数学的アルゴリズムを使用しています。
最新版はラインクロスしたりエリア侵入したりなどのイベント発生時にオーディオを再生します。
コードを簡素化したテストプログラムも追加しました。ライン交差やエリア侵入判定関数の使い方などを学ぶことが可能です。(test-line-cross-area-intrusion.py)

[ Update - 04-Apr-2022 ] OpenVINO API 2.0 support. Program is now compatible with OpenVINO 2022.1 (and not compatible with OpenVINO 2021 and older ones)

Object Tracking and Line Crossing Demo

Required DL Models to Run This Demo

The demo expects the following models in the Intermediate Representation (IR) format:

For person / pedestrian detection and re-identification
- pedestrian-detection-adas-0002
- person-reidentification-retail-0031

You can download these models from OpenVINO Open Model Zoo. In the models.lst is the list of appropriate models for this demo that can be obtained via Model downloader. Please see more information about Model downloader here.

How to Run

0. Prerequisites

OpenVINO 2021.3
- If you haven't installed it, go to the OpenVINO web page and follow the Get Started guide to do it.

1. Install dependencies

The demo depends on:

opencv-python
numpy
scipy
munkres

To install all the required Python modules you can use:

(Linux) pip3 install -r requirements.txt
(Win10) pip install -r requirements.txt

2. Download DL models from OMZ

Use Model Downloader to download the required models and convert the downloaded model into OpenVINO IR models with Model Converter.

(Linux) python3 $INTEL_OPENVINO_DIR/deployment_tools/tools/model_downloader/downloader.py --list models.lst
(Win10) python "%INTEL_OPENVINO_DIR%\deployment_tools\tools\model_downloader\downloader.py" --list models.lst

3. Run the demo app

(Linux) python3 object-detection-and-line-cross.py
(Win10) python object-detection-and-line-cross.py

Demo Output

The application draws the results on the screen.

Simple test program

test-line-cross-area-intrusion.py is a simple test program for line-crossing and area-intrusion detection. You can move mouse cursor to test how it works.

Coordinates checker

You can utilize coordinates-checker to get the coordinates of point of interest to define boundary lines and area polygons. The coordinates-checker is also included in this project as a submodule. See coordinates-checker GitHub repo for details.
python coordinates-checker\coordinates-checker.py [image.jpg / movie.mp4 / 0 ]

Tested Environment

Windows 11 x64
Intel(r) Distribution of OpenVINO(tm) toolkit 2022.1
Python 3.7 x64

object-tracking-line-crossing-area-intrusion's People

Contributors

Stargazers

Watchers

object-tracking-line-crossing-area-intrusion's Issues

执行object-detection-and-line-cross.py

File "object-detection-and-line-cross.py", line 11, in
from openvino.preprocess import PrePostProcessor, ResizeAlgorithm
ModuleNotFoundError: No module named 'openvino.preprocess'、

谁能教教我

from .ie_api import * importerror dll load failed

Why when i run my own py project get error : from .ie_api import * importerror dll load failed

Unnecessary actions in angle caclulation

Hello! Your repo is very helpful for me!
But I find one unnecessary action in your code. Your func calcVectorAngle is unnecessary
Your code in line_boundary_check.py:

def calcVectorAngle( point1, point2, point3, point4 ):
  u = np.array(line_vectorize(point1, point2))
  v = np.array(line_vectorize(point3, point4))
  i = np.inner(u, v)
  n = LA.norm(u) * LA.norm(v)
  c = i / n
  a = np.rad2deg(np.arccos(np.clip(c, -1.0, 1.0)))
  if u[0]*v[1]-u[1]*v[0]<0:
    return a
  else:
    return 360-a

Your code in test-line-cross-area-intrusion.py:

    if intersect == True:
        angle = calcVectorAngle(traj_p0, traj_p1, bLine_p0, bLine_p1)   # Calculate angle between trajectory and boundary line
        if angle<180:
            boundary_line.count1 += 1
        else:
            boundary_line.count2 += 1

What I suggest:

def calc_orientation(point1, point2, point3, point4):
    u = np.array(line_vectorize(point1, point2))
    v = np.array(line_vectorize(point3, point4))
    return u[0] * v[1] - u[1] * v[0] < 0

    if intersect == True:
        if calc_orientation(traj_p0, traj_p1, bLine_p0, bLine_p1):
            boundary_line.count1 += 1
        else:
            boundary_line.count2 += 1

No Model downloader Found

On opening the Model Downloader , the page shows error404

openvino2021.4.2 versus openvino2022.1.0

@yas-sim
Thanks for creating this project. I managed to get your project working in both openvino2021.4.2 and openvino2022.1.0 using a podman container by roughly following the install documented here https://github.com/balena-io-examples/balena-OpenVino/blob/master/openvino/Dockerfile.template

It appears that the project works better with commit a0de2df (after I changed GPU to CPU) and it does not work as well with the current HEAD and openvino2022.1.0 -- the tracks are messed up. I am not sure if I am doing something wrong

openvino

Exception: Path to the model intel/pedestrian-detection-adas-0002/FP16/pedestrian-detection-adas-0002.xm1doesn't exist or it's a directory
This file exists in my folder，but still getting error

Getting error while running object-detection-and-line-cross.py

Hi,
I have installed all the requirements, when i ran object-detection-and-line-cross.py i'm getting this error

  File "object-detection-and-line-cross.py", line 330, in <module>
    sys.exit(main() or 0)
  File "object-detection-and-line-cross.py", line 233, in main
    net_det  = ie.read_network(model_det+'.xml', model_det+'.bin')           # model=pedestrian-detection-adas-0002
  File "ie_api.pyx", line 357, in openvino.inference_engine.ie_api.IECore.read_network
  File "ie_api.pyx", line 400, in openvino.inference_engine.ie_api.IECore.read_network
RuntimeError: Unknown model format! Cannot find reader for model format: xml and read the model: intel/pedestrian-detection-adas-0002/FP16/pedestrian-detection-adas-0002.xml. Please check that reader library exists in your PATH.

i have downloaded models too but How can i solve this?

Similarity is not working

I have checked the cos_sim_matrix is always filled with zeros. So even if you change the thresold nothing will happen. The id is changing only from timeout.

Error: ModuleNotFoundError: No module named 'pyaudio'

Dear Author,
When I am running the codes python object-detection-and-line-cross.py and python audio_playback_bg.py, I am getting the following error.

File "C:\Users\Ishan.Borkar\Documents\October 2021\21-10-2021(object tracking line crossing)\object-tracking-line-crossing-area-intrusion-master\audio_playback_bg.py", line 3, in
import pyaudio
ModuleNotFoundError: No module named 'pyaudio'.

I tried to perform pip install pyaudio, still the error is not getting resolved.
Kindly help me in solving this doubt. Awaiting your response.

yas-sim / object-tracking-line-crossing-area-intrusion Goto Github PK