Giter VIP home page Giter VIP logo

hailo-ai / tappas Goto Github PK

View Code? Open in Web Editor NEW
42.0 3.0 28.0 442.79 MB

High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices

Home Page: http://hailo.ai/

License: GNU Lesser General Public License v2.1

Shell 15.82% Python 5.07% C++ 75.94% Meson 2.29% C 0.88%
deep-learning edge-ai gstreamer hardware-acceleration machine-learning neural-networks video-processing

tappas's Introduction

Hailo TAPPAS - Optimized Execution of Video-Processing Pipelines

image


Overview

TAPPAS is Hailo's set of full application examples, implementing pipeline elements and pre-trained AI tasks.

Demonstrating Hailo's system integration scenario of specific use cases on predefined systems (software and Hardware platforms). It can be used for evaluations, reference code and demos:

  • Accelerating time to market by reducing development time and deployment effort
  • Simplifying integration with Hailo’s runtime SW stack
  • Providing a starting point for customers to fine-tune their applications

image


Getting Started

Prerequisites

  • Hailo-8 device
  • HailoRT PCIe driver installed
  • At least 6GB's of free disk space

Note

This version is compatible with HailoRT v4.17.

Installation

Option Instructions Supported OS
Hailo SW Suite* SW Suite Install guide Ubuntu x86 20.04, Ubuntu x86 22.04
Pre-built Docker image Docker install guide Ubuntu x86 20.04, Ubuntu x86 22.04, Ubuntu aarch64 20.04 (64-bit)
Manual install Manual install guide Ubuntu x86 20.04, Ubuntu x86 22.04, Ubuntu aarch64 20.04
Yocto installation Read more about Yocto installation Yocto supported BSP's
Raspberry pi installation Read more about Raspberry pi installation Raspberry Pi OS, Ubuntu 20.04

* It is recommended to start your development journey by first installing the Hailo SW Suite

Documentation


Example Applications Built with TAPPAS

TAPPAS comes with a rich set of pre-configured pipelines optimized for different common hosts.

Important

* All example applications utilize both the host (for non-neural tasks) and the Neural-Network Core (for neural-networks inference), therefore performance results are affected by the host. * General application examples do not include any architecture-specific accelerator usage, and therefore will provide the easiest way to run an application, but with sub-optimal performance. * Architecture-specific application examples (i.MX, Raspberry PI, etc.) use platform-specific hardware accelerators and are not compatible with different architectures.

Note

All i.MX example application are validated on i.MX8 and i.MX6 platforms and are compatible with the architectures.

Note

Running application examples requires a direct connection to a monitor.

Basic Single Network Pipelines

Pipelines that run a single network. The diagram below shows the pipeline data-flow.

image

The following table details the currently available examples.

General i.MX8 RPi4 x86 Hardware Accelerated Rockchip
Object Detection check_mark check_mark check_mark check_mark
Depth Estimation check_mark check_mark check_mark
Instance segmentation check_mark
Classification with Python Postprocessing check_mark
Object Detection Multiple Devices (Century) check_mark check_mark
Face Recognition check_mark

Two Network Pipelines

Examples of basic pipelines running two networks. The parallel networks pipeline is an simple extension of the single network pipeline and is shown in the following diagram:

image

The cascaded (serial) flow shows two networks running in series. This example pipeline is of the popular configuration where the first network is a detector which finds some Region-of-Interest (ROI) in the input image and the second network processes the cropped ROI (a face-detection-and-landmarking use case of this pipeline is shown at the top of this guide). The pipeline is shown in the following diagram:

image

General i.MX8 RPi4 x86 Hardware Accelerated Rockchip
Cascaded - Face Detection & Landmarks check_mark check_mark
Cascaded - Person Det & Single Person Pose Estimation check_mark check_mark
Cascaded - Face Detection & Recognition check_mark

Multi-Stream Pipelines

image

General i.MX8 RPi4 x86 Hardware Accelerated Rockchip
Multi-stream Object Detection check_mark check_mark check_mark
Multi-stream Multi-Device Object Detection check_mark

Pipelines for High-Resolution Processing Via Tiling

image

General i.MX8 RPi4 x86 Hardware Accelerated Rockchip
HD Object Detection check_mark check_mark

Example Use Case Pipelines

Our LPR reference application demonstrates the use of 3 networks, with a database. The pipeline demonstrates inference based decision making (Vehicle detection) for secondary inference tasks (License plate data extraction). This allows multiple networks to cooperate in the pipeline for reactive behavior.

image

Our Multi-Person Multi-Camera Tracking reference application demonstrates person tracking across multiple streams using RE-ID tracking. The pipeline demonstrates another method for inference based decision making that also connects between different video streams.

image

Our VMS reference application demonstrates the use of 5 networks over multiple streams, and is a reference for video management system applications.

image

General i.MX8 RPi4 x86 Hardware Accelerated Rockchip
LPR check_mark check_mark check_mark
RE-ID check_mark

Changelog

v3.28.0 (April 2024)

  • Tappas was updated in this version, with a revised list of supported platform and apps
  • Added yolov8 (as default) to Detection application examples
  • Fixed various stability issues across apps and platforms
  • This release is aligned to Hailo-15 Vision Processor Software Package 2024-04

v3.27.2 (March 2024)

  • Aligned to Hailo-15 Vision Processor Software Package 2024-01.2
  • Added a new example application - Frontend Update
  • Updated the hailo OSD API
  • Detection application now works with an updated hailonet element
  • Various bugs fixes

v3.27.1 (February 2024)

  • Aligned to Hailo-15 Vision Processor Software Package 2024-01.1
  • Added a new Hailo-15 external host application that saves udp stream to file

v3.27.0 (January 2024)

  • Updated the Hailo-15 applications to use the updated Media Library implementation:
    • Basic Security Camera (streaming)
    • Detection
    • Single Stream OSD (On-Screen Display)
  • Added a folder for external host scripts and added the UDP Stream Display script

Note

TAPPAS supports both Hailo-15 and Hailo-8. Temporarily, in this version, only the following Hailo-8 based example applications are supported:

  • Detection
    • yolov5
    • mobilenet_ssd
  • Multi-Stream Detection
    • Multi-Stream Detection
    • MultiStream Detection with Stream Multiplexer
  • License Plate Recognition

These applications are supported under the general folder (x86-based platforms).

v3.26.2 (December 2023)

  • Aligned to Hailo-15 Vision Processor Software Package 2023-10.2
  • Add grayscale support for Media Library Front-end
  • Various bug fixes for Hailo-15 pipelines

v3.26.1 (November 2023)

  • Aligned to Hailo-15 Vision Processor Software Package 2023-10.1
  • Updated OSD configuration to support new dynamic features and adjust to the updated Media Library implementation
  • Added a script for displaying UDP streams
  • Basic security camera (Media Library implementation) now support 5 outputs

v3.26.0 (October 2023)

  • Added Hailo-15 supported application examples:
    • Detection
    • License Plate Recognition
    • A new Hailo-15 specific example application - Basic Security Camera (streaming)
  • Removed Yolact models support from Instance Segmentation
  • Various bug fixes:
    • Fixed the Detection application on i.MX6 platforms
    • Fixed an issue with Face Recognition which prevented faces to be recognized ins some scenarios
    • Fixed an issue which caused a warning when running some networks

v3.25.0 (July 2023)

  • Improved Yolov5seg post-process performance
  • Updated Yolo networks to use the HailoRT native post-process (selected models)
  • Added "non-blocking mode" and "wait-time" properties to hailoroundrobin element

v3.24.0 (March 2023)

  • Added support for Rockchip RK3588 (validated on Firefly ITX-3588J platform)
  • Video Management System now supports multi-device (Ubuntu 22.04 only)
  • Video Management System (single device) now works on Ubuntu 20.04
  • Added a new model to Instance Segmentation Pipeline:
    • yolov5seg - which has improved performance compared to yolact
  • New applications for i.MX8:
    • Object Detection and Pose Estimation (cascaded)
    • Multi-Stream Detection
  • Added a TAPPAS Graphic User Interface to easily run selected general example applications (preview) on the TAPPAS Docker - to activate it, run tappas-gui
  • Added back yolox_l_leaky to the Century general application
  • Reduced docker size

v3.23.1 (February 2023)

v3.23.0 (December 2022)

v3.22.0 (November 2022)

  • New element hailoimportzmq - provides an entry point for importing metadata exported by hailoexportzmq (HailoObjects) into the pipeline
  • Added Depth Estimation, Object Detection and Classification pipelines for i.MX6 Pipelines
  • Changed the debugging tracers to use an internal tracing mechanism

v3.21.0 (October 2022)

  • New Apps:
  • New elements - hailoexportfile and hailoexportmq which provide an access point in the pipeline to export metadata (HailoObjects)
  • Improved pipeline profiling by adding new tracers and replacing the GUI of gst-shark
  • Ubuntu 22 is now supported (GStreamer 1.20, preview)
  • Yocto Kirkstone is now supported (GStreamer 1.20)

v3.20.0 (August 2022)

  • New Apps:
  • Improvements to Multi-Camera Multi-Person Tracking (RE-ID) pipeline (released)

v3.19.1 (July 2022)

v3.19.0 (June 2022)

  • New Apps:
    • Added Cascading networks, Depth Estimation, Pose Estimation and Semantic Segmentation pipelines for i.MX Pipelines
  • Added an option to control post-process parameters via a JSON configuration for the detection application
  • Added support for Raspberry Pi Raspbian OS
  • Native Application now uses TAPPAS post-process
  • LPR (License Plate Recognition) pipeline is simplified to bash only
  • New detection post-process - Nanodet

Note

Ubuntu 18.04 will be deprecated in TAPPAS future version

Note

Python 3.6 will be deprecated in TAPPAS future version

v3.18.0 (April 2022)

  • New Apps:
    • LPR (License Plate Recognition) pipeline and facial landmark pipeline for i.MX Pipelines
  • Added the ability of compiling a specific TAPPAS target (post-processes, elements)
  • Improved the performance of Raspberry Pi example applications

v3.17.0 (March 2022)

  • New Apps:
    • LPR (License Plate Recognition) pipeline for General Pipelines (preview)
    • Detection & pose estimation app
    • Detection (MobilenetSSD) - Multi scale tiling app
  • Update infrastructure to use new HailoRT installation packages
  • Code is now publicly available on Github

v3.16.0 (March 2022)

  • New Apps:
    • Hailo Century app - Demonstrates detection on one video file source over 6 different Hailo-8 devices
    • Python app - A classification app using a post-process written in Python
  • New Elements:
    • Tracking element "HailoTracker" - Add tracking capabilities
    • Python element "HailoPyFilter" - Enables to write post-processes using Python
  • Yocto Hardknott is now supported
  • Raspberry Pi 4 Ubuntu dedicated apps
  • HailoCropper cropping bug fixes
  • HailoCropper now accepts cropping method as a shared object (.so)

v3.14.1 (March 2022)

  • Fix Yocto Gatesgarth compilation issue
  • Added support for hosts without X-Video adapter

v3.15.0 (February 2022)

  • New Apps:
    • Detection and depth estimation - Networks switch app
    • Detection (MobilenetSSD) - Single scale tilling app

v3.14.0 (January 2022)

  • New Apps:
    • Cascading apps - Face detection and then facial landmarking
  • New Yocto layer - Meta-hailo-tappas
  • Window enlargement is now supported
  • Added the ability to run on multiple devices
  • Improved latency on Multi-device RTSP app

v3.13.0 (November 2021)

  • Context switch networks in multi-stream apps are now supported
  • New Apps:
    • Yolact - Instance segmentation
    • FastDepth - Depth estimation
    • Two networks in parallel on the same device - FastDepth + Mobilenet SSD
    • Retinaface
  • Control Element Integration - Displaying device stats inside a GStreamer pipeline (Power, Temperature)
  • New Yocto recipes - Compiling our GStreamer plugins is now available as a Yocto recipe
  • Added a C++ detection example (native C++ example for writing an app, without GStreamer)

v3.12.0 (October 2021)

  • Detection app - MobilenetSSD added
  • NVR multi-stream multi device app (detection and pose estimation)
  • Facial Landmarks app
  • Segmentation app
  • Classification app
  • Face detection app
  • Hailomuxer gstreamer element
  • Postprocess implementations for various networks
  • GStreamer infrastructure improvements
  • Added ARM architecture support and documentation

v3.11.0 (September 2021)

  • GStreamer based initial release
  • NVR multi-stream detection app
  • Detection app
  • Hailofilter gstreamer element
  • Pose Estimation app

tappas's People

Contributors

hailort-automation avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.