Giter VIP home page Giter VIP logo

log-ingesting-api's Introduction

Log Access Analytics

Foi utilizado o Logstash em conjunto com o Elasticsearch para realizar o tratamento, armazenamento e métricas dos logs. O framework utilizado na API foi o Quarkus. Quando o serviço recebe um log via REST, este é salvo no Logstash, que realiza o tratamento do timestamp para extrair a semana do ano, dia do mês, ano e minuto em que o log ocorreu. Após o tratamento isto é enviado para o Elasticsearch para ser indexado. E no momento de gerar métricas, foram utilizados os Terms Aggregations do Elasticsearch.

Configuração utilizada no Logstash

input {
	http { 
          port => 8090
       }
}

filter {
  json{
    	source => "message"
	}
  date { match => ["accessTimestamp", "yyyy-MMMM-dd HH:mm:ss", "UNIX_MS"]
    	target => "@timestamp"
    	add_field => {"minute" => "%{+mm}"}
    	add_field => {"week" => "%{+w}"}
    	add_field => {"day" => "%{+dd}"}
    	add_field => {"year" => "%{+yyyy}"}
  	}
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
}

log-ingesting-api's People

Contributors

leonp967 avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.