Giter VIP home page Giter VIP logo

whisper's People

Contributors

dependabot[bot] avatar k-abram avatar kabramcs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

whisper's Issues

Questions about intended behavior

Guys, this looks like a great useful project. I am testing it out and wanting to confirm my understanding on what the expected behavior is.

  • Which messages will be potentially coalesced? Is it based on the 'canonical' message (ie message without parameters)? Ie. of the following two examples will the former be coalesced with itself, but the latter not? In either case, would affect coalescing if the user was the same or varied?
LOGGER.error("Error processing user {} request ", user);

vs.

LOGGER.error("Error processing user "+user+" request ");

Or is it the case that you may suppress any/all messages even if they are unrelated?

  • How much context is combined into a single email? I've noticed that the emails contain more than one log message. Assuming we are only talking about behavior BEFORE suppression - will I get one email per log message? Or fewer? And will there be context attached to the email - like one can get via CyclicBufferTracker?

Process keeps running?

I'm trying to run the below simple Java code

package chapters.configuration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;


public class MyApp1 {
  final static Logger logger = LoggerFactory.getLogger(MyApp1.class);

   public static void main(String[] args) {

      for(int i=0;i<10;i++){
          logger.error("NPE", new NullPointerException());
  }
  for(int i=0;i<10;i++){
    logger.error("Exiting application."+i);
  }
  }
}

using the logback configuration file

<?xml version="1.0" encoding="UTF-8" ?>    
<configuration scan="true" scanPeriod="1 minute" debug="true">
    <contextName>testContext</contextName>

    <!-- Console Appender -->
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{ApplicationId}] [%X{UnitOfWorkId}] [%thread] %-5level %logger{36} - %msg%n</pattern>
        </layout>
    </appender>

    <appender name="errorEmail" class="ch.qos.logback.classic.net.SMTPAppender">
        <SMTPHost>localhost</SMTPHost>
        <SMTPPort>25</SMTPPort>

        <to>[email protected]</to>
        <from>test@localhost</from>

        <subject>%logger{20} - %m</subject>

        <layout class="ch.qos.logback.classic.PatternLayout">
            <pattern>%date %-5level %logger{35} - %message%n</pattern>
        </layout>
    </appender>

        <!-- This is the whisper appender that routes error logs to the errorEmail appender -->
    <appender name="whisper"
        class="com.eclecticlogic.whisper.logback.WhisperAppender">
        <!-- Filter out non error logs -->
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
        <!-- This is the name of the logging category to use to send out error digests. This is associated with a 
            specific smtp appender below. -->
        <digestLoggerName>digest.appender.logger</digestLoggerName>
        <!--  suppressAfter specifies the criteria to enter suppression. The example below says that if 3 errors of the same kind
        are encountered within a 5 minute window, then suppression should kick in. -->
        <suppressAfter>3 in 5 minutes</suppressAfter>
        <!-- expireAfter specifies how much of silence the logger should see for the error message being suppressed 
        before stopping suppression. --> 
        <expireAfter>4 minutes</expireAfter>
        <!-- digestFrequency specifies how often error email digests should be sent containing statistics on messages 
        suppressed -->
        <digestFrequency>10 seconds</digestFrequency>

        <!-- The pass-through appender for the normal case when suppression is not in-force. -->
        <appender-ref ref="errorEmail" />
    </appender>


    <appender name="digest" class="ch.qos.logback.core.ConsoleAppender">
        <!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder 
            by default -->
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %X{whisper.digest.subject} - %msg%n</pattern>
        </encoder>
    </appender>

    <!-- The logger name for digest is associated with a specific appender. It is important that additivity be 
         set to false otherwise this will also show up in the regular smtp appender and log file. -->
    <logger name="digest.appender.logger" level="error" additivity="false">
        <appender-ref ref="digest" />
    </logger>

    <root level="info">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="whisper" />
    </root>
</configuration>

I was expecting the app to exit after logging "Exiting application.9", but it hangs around like for ever.

Suppression only ends if another message is sent in

Maybe I am misunderstanding the functionality, but in my mind, a suppression log message indicating that no messages have been suppressed is not intended behavior. As a look at the Muffler.java file the only place where suppression is turned off is during log, which indicates that it will not turn off until another duplicate message is sent. In our use case we have a five minute burst of messages which turns on suppression, but the message is not seen again after that burst, and still our email system is sent a report every 20 minutes indicating that 0 messages have been suppressed. I would think two things need to happen.

First, allow the system to be customizable so that suppression log messages do not need to be sent if nothing is suppressed.

Two, check to turn off suppression during digest.

My version of digest is below.

/**
 * Record message suppression digest.
 * 
 * @param digest
 */
public void digest(final Digest digest) {
    // Write digest only if we've suppressed something. Without the null check, an error message that is written
    // just once will cause a NPE (see https://github.com/eclecticlogic/whisper/issues/1)
    final Message<E> m = this.lastMessage.get();
    if (m != null) {
        // Added check to see if any messages have been supressed since last digest otherwise
        // code sends suppression message indicating nothing happened
        if (this.messagesSinceLastDigest.get() == 0) {
            if (m.getMessageAge() > this.manager.getSuppressionExpirationTime()) {
                // Suppression ends
                this.inSuppression = false;
                this.manager.remove(this);
                this.lastMessage.set(null);
            }
        } else {
            final DigestMessage dm = new DigestMessage();
            dm.setMessage(m.getMessage());
            dm.setFullMessage(m.getFullMessage());
            dm.setMessagesSinceLastDigest(this.messagesSinceLastDigest.get());
            dm.setMessagesSinceSuppressionStart(this.messagesSinceSuppressionStart.get());
            this.messagesSinceLastDigest.set(0);
            digest.add(dm);
        }
    }
}

Potential NPE in Muffler#digest when Message is not yet set

I had the following Exception with Version 1.0.0

Exception in thread "Timer-0" java.lang.NullPointerException
    at com.eclecticlogic.whisper.core.Muffler.digest(Muffler.java:88)
    at com.eclecticlogic.whisper.core.WhisperManager.run(WhisperManager.java:122)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)

This is a race condition and is possible when the TimerThread is running between queuesByMessage.putIfAbsent and muffler.log in WispherManager#log: then lastMessage is not yet set.
I think it is enough to add a check for a Null Message in Muffler#digest

NPE because message is null.

I ran into the following NullPointerException:

16:40:01,170 |-ERROR in com.eclecticlogic.whisper.logback.WhisperAppender[whisper] - Appender [whisper] failed to append. java.lang.NullPointerException
at java.lang.NullPointerException
at at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at at com.eclecticlogic.whisper.core.WhisperManager.log(WhisperManager.java:82)
at at com.eclecticlogic.whisper.logback.WhisperAppender.append(WhisperAppender.java:90)
at at com.eclecticlogic.whisper.logback.WhisperAppender.append(WhisperAppender.java:37)

I back tracked that to the original error message (logged by the File Appender):

14-01 16:40:01 [ajp-bio-8812-exec-12] com.hcs.data.dao.jdbc.util.JDBCDAOUtil ERROR null
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'webtl_orc_dev_1_4_3.procedures' doesn't exist

Which came from this code in my application:

   } catch (SQLException ex) {
        log.log(Level.SEVERE, null, ex);
    }

While you can make a good argument that it is poor form to provide a null message, it does not cause problems for Logback's File Appender or SMTP Appender. Even if the WhisperAppender didn't throw an NPE, it would cause problems for the digest if I have multiple different "errors" being logged with a null message.

So, my question/bug/concern is two fold:

  1. Is there a better way to handle this then throwing an NPE when the message is null?
  2. Is there a better way to recognize duplicate messages that have null messages?

I can fix my code, of course, but it is easy enough for another null message to creep back in at a later date (old programming habits can die hard), which could cause us to miss important error messages.

This may or may not be a bug, but I couldn't find anywhere else to raise the question, so my apologies if this was the wrong spot.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.