Log-analyzer is a human and straightforward log analyzer for rapid troubleshooting.

Note:

This project starts to be an interview project. Now I am using it to monitor my website traffic.

pip install git+http://github.com/yurilaaziz/log-analyzer.git

pip install log-analyzer

log-analyzer 

By default, the log-analyzer reads the log from /var/log/access.log

LOGANALYZER_CONFIG_FILE=sample.config.yml log-analyzer 

LOGANALYZER_PARSER_INPUT=/tmp/test.log log-analyzer 

Log Analyzer containers on three components :

• Parser Process :

  • Reads the log file
  • Process lines following a pattern class
  • Push data to the Persistence driver

• Consumer Process (Console Display):

  • Reads data from the persistence driver
  • Display an array ordered by hits

• Alert Manager:

  • Compute rules defined in the pattern alert's class
  • Push notification to persistence driver to be displayed on the console

• Persistence Driver:

  • Define interfaces between producer and consumer.
  • Allow persisting data to different data-store

Log Analyzer uses Config42 to manage its configuration. that means every variable present in the configuration file could be overloaded from Environment variables, Configuration file, external data store (ETCD)

Here an example of default configuration in YAML format: configuration file

• Rework the Alert Manager process to Delete Alert Class and read Alerting rules from the configuration
• Rework the main for a better CLI with 'docopt'
• Write a persistence driver to support ElasticSearch/InfluxDB data-store
• Support multiple log files/sources
• Support multiple Persistence drivers