The automation of detecting anomalous events in videos is a challenging problem that currently attracts a lot of attention by researchers, but also has broad applications across industry verticals.
The approach involves training deep neural networks to develop an in-depth understanding of the physical and causal rules in the observed scenes. The model effectively learns to predict future frames in the video in a self-supervised fashion.
By calculating the error in this prediction, it is then possible to detect if something unusual, an anomalous event, occurred, if there is a large prediction error.
The approach can be used both in a supervised and unsupervised fashion, thus enabling the detection of pre-defined anomalies, but also of anomalous events that have never occurred in the past.
Post on LinkedIn (includes video demonstration)
You will learn:
- How to adapt an existing neural network architecture to your use-case.
- How to prepare video data for deep learning.
- How to perform hyperparameter tuning with HyperDrive to improve the performance of you model.
- How to deploy a deep neural network as a webservice for video processing.
- How to post-process the output of a Keras model for secondary tasks (here, anomaly detection)
- How to define a build pipeline for DevOps.
- Some familiarity with concepts and frameworks for neural networks:
- Framework: Keras
- Concepts: convolutional, recurrent, and pooling layers.
- Knowledge of basic data science and machine learning concepts. Here and here you'll find short introductory material.
- Moderate skills in coding with Python and machine learning using Python. A good place to start is here.
- Various python modules. We recommend working with a conda environement (see
environment.yml
) - Documentation - VS code https://code.visualstudio.com/
- X2Go https://wiki.x2go.org/doku.php
We found that a useful development environment is to have a VM with a GPU and connect to it using X2Go.
A computer with a GPU, Standard NC6 sufficient, faster learning with NC6_v2/3 or ND6. compare VM sizes
UCSD Anomaly Detection Dataset
- Transfer learning - How to quickly retrain the model on new data.
- AML Pipelines - Use AML pipelines to scale your solution.
- MLOps - How to quickly scale your solution with the MLOps extension for DevOps.
-
Research Article: Deep predictive coding networks for video prediction and unsupervised learning by Lotter, W., Kreiman, G. and Cox, D., 2016.
@article{lotter2016deep, title={Deep predictive coding networks for video prediction and unsupervised learning}, author={Lotter, William and Kreiman, Gabriel and Cox, David}, journal={arXiv preprint arXiv:1605.08104}, year={2016} }
-
Original Prednet implentation is on github.com. Note, that the original implementation will only work in Python 2, but not in Python 3.
-
Interesting blog post on Self-Supervised Video Anomaly Detection by Steve Shimozaki