An anomaly is defined as something which is different from the normal. Using supervised techniques for anomaly detection is not a good choice, because we have a lot of example cases for a normal event, but the abnormal event happens rarely.Imagine using 10 sec clips of video feed, and classifying it as normal or abnormal(may be an accident), in this case an accident happens rarely, so the dataset would have a lot of normal clips and very few abnormal clips, so the dataset is imbalanced, also in the case of a huge dataset hand labelling all these data instances would be difficult. So the wisest choice to detect anomalies is to learn the features that make a particular event normal and then using these we need to be able to detect the anomalies.Some ways to learn the normal representations are:
- training an autoencoder using the normal data and using the reconstruction error as an anomaly metric.
- in the case of time series data, predicting the next instance and using the prediction error as an anomaly metric.
[UCSD Dataset] - http://www.svcl.ucsd.edu/projects/anomaly/dataset.html