This tool parses log data and allows to define analysis pipelines for anomaly detection. It was designed to run the analysis with limited resources and lowest possible permissions to make it suitable for production server use.

In order to install logdata-anomaly-miner a Linux system with python >= 3.6 is required. Debian-based distributions are currently recommended.

See requirements.txt for further module dependencies

There are Debian packages for logdata-anomaly-miner in the official Debian/Ubuntu repositories.

apt-get update && apt-get install logdata-anomaly-miner

The following command will install the latest stable release:

cd $HOME
wget https://raw.githubusercontent.com/ait-aecid/logdata-anomaly-miner/main/scripts/aminer_install.sh
chmod +x aminer_install.sh
./aminer_install.sh

For installation with Docker see: Deployment with Docker

Here are some resources to read in order to get started with configurations:

• Getting started
• Some available configurations
• Documentation
• Wiki

Publications and talks:

• Wurzenberger M., Skopik F., Settanni G., Fiedler R. (2018): AECID: A Self-learning Anomaly Detection Approach Based on Light-weight Log Parser Models. 4th International Conference on Information Systems Security and Privacy (ICISSP 2018), January 22-24, 2018, Funchal, Madeira - Portugal. INSTICC. [PDF]
• Wurzenberger M., Landauer M., Skopik F., Kastner W. (2019): AECID-PG: AECID-PG: A Tree-Based Log Parser Generator To Enable Log Analysis. 4th IEEE/IFIP International Workshop on Analytics for Network and Service Management (AnNet 2019) in conjunction with the IFIP/IEEE International Symposium on Integrated Network Management (IM), April 8, 2019, Washington D.C., USA. IEEE. [PDF]
• Landauer M., Skopik F., Wurzenberger M., Hotwagner W., Rauber A. (2019): A Framework for Cyber Threat Intelligence Extraction from Raw Log Data. International Workshop on Big Data Analytics for Cyber Threat Hunting (CyberHunt 2019) in conjunction with the IEEE International Conference on Big Data 2019, December 9-12, 2019, Los Angeles, CA, USA. IEEE. [PDF]

A complete list of publications can be found at https://aecid.ait.ac.at/further-information/.

We're happily taking patches and other contributions. Please see the following links for how to get started:

• How to install a development environment
• Git development workflow

If you encounter any bugs, please create an issue on Github.

If you discover any security-related issues read the SECURITY.md first and report the issues.

GPL-3.0