procaseycash / log-manager Goto Github PK

This application can be built using a microservice architecture such that all modular system are built as separate service while having the logManager as a single service system that house the routing strategy to the other microservices system.
Considering the kind of architecture. each service will either have a unique route pattern such as `users/`, `adminitrattions/` `payments/`, these patterns can be used to denote each service system and able to detect which log activities belongs to a particular microservice or having a unique serviceId present in every request header of that service.
In this case, I will be using a structural logging over file system in order to enable ease in data querying and mining.

There are several log management tools such as
    1. Loggly
    2. Logstash
    3. Splunk
    4. PaperTrails
    5. Storm
    6. Graylog2
 

 The major overall issues is in 
     1. volume of data
     2. Price
     3. Security regarding sensitive data.
 
 These tools are capable in logging resources but there are one or more issues to lookout for.  
    1. Loggly: This is a tools mainly for devops to parse data coming from your app servers. Anything beyond that you’ll have to build yourself.
    2. Logstash: This is stack because it entails one or more technology such as kibana and elastic search. these stacks requires configuring separate instance on production which brings more management and complexity
    3. Splunk: For high scalability, requires dedicated cluster and somewhat subjective which means the cost of running it is high and it is also complex coming to management.
    4. PaperTrails: This is mosty a text based logging system which gives complexity for proper and advance data mining.
    5. Storm: This is limiting in amount of data that can be sent.
    6. Graylog2: The newest among others, this makes it limiting in usage.
        

The only alternative tools I think is best if we are more concerned about high level of customization and proprietary changes is `Build it yourself` and I would suggest the use of
    1. Document DB such MongoDB
    2. Elastic Search for deep querying and analysis.

1  I adopted the use of `serviceId` as an identify for each microservice
2. This `serviceId` is predefined ID for each micro service and these are setup in env variables and each pointing to their baseurl.
3. I then make available a route strategy to pass request to the expected microservice
4. The interceptor is used to carry the main processing of logging by using a cloud support `Loggly`.
5. Configuration params for loggly can be found in env
6. This interceptor helps to log request after a complete cycle to loggly and able to catch all errors that might occur.

1. Implementation of node clustering to enable efficiency for concurrent request scaling.

Due to time and load of work at work. I took less than 24hrs to build to this stage of the app.
So, I was unable to do the followings.
1. Docker for containerization.
2. Node Clustering Implementation.
3. Test Cases.