Giter VIP home page Giter VIP logo

winston-logstash's Introduction

winston-logstash

Build Status

Integration tests with Logstash instance

A Logstash TCP transport for winston.

Usage

Winston 2.x

// See test cases from test-bench/winston-2x/test/smoke.js
const winston = require("winston");
const transports = require("winston-logstash");

const logger = new winston.Logger({
  transports: [
    new transports.Logstash({
      port: 28777,
      node_name: "my node name",
      host: "127.0.0.1",
    }),
  ],
});

logger.info("Hello world!");

Winston 3.x

// See test cases from test-bench/winston-3x/test/smoke.js
const winston = require("winston");
const LogstashTransport = require("winston-logstash/lib/winston-logstash-latest");

const logger = winston.createLogger({
  transports: [
    new LogstashTransport({
      port: 28777,
      node_name: "my node name",
      host: "127.0.0.1",
    }),
  ],
});

logger.info("Hello world!");

Logstash config

# See example from test-bench/logstash/logstash/pipeline/default.conf
input {
  # Sample input over TCP
  tcp { port => 28777 type=>"sample" }
}
output {
  stdout { debug => true }
}

filter {
  json {
    source => "message"
  }
}

FAQ

Error handling with the transport

While this is a relatively complex transport with an internal state and IO, I think the current solution is the yet best approach to network failure. Transport is transparent about the errors and lets the user decide what to do in the case of an error. Suppose the developer chooses not to address them, it fallback to a safe default, an exception that stops the process. I think this way; it's simple but not always easy.

You can check the test case from test-bench folder where is the test case per Winston's version. The simplest ways to write the error logic would be:

Winston 2.x

For Winston 2.x you have to add the error listener to the transport.

const logstashTransport =  new LogstashTransport({...});

logstashTransport.on('error', (error) => {
  // Make the decission in here
  throw new Error('Stop the press, logging not working');
});

Winston 3.x

For Winston 3.x you have to add the error listener to the logger. Remember that there might be also other errors which are not originated from LogstashTransport.

const logger = winston.createLogger({
      transports: [
        new LogstashTransport({
              ...
               max_connect_retries: -1
              ...
              })]});

logger.on('error', (error) => {
  // Make the decission in here
  throw new Error('Stop the press, logging not working');
});

What configuration options are available?

See documentation from docs/configuration

How to keep the connection open while Logstash is restarting?

It's possible to set max_connect_retries to -1 (infinite) so the client keeps trying to connect to the Logstash. So when Logstash is restarted the retry logic will reconnect when it comes back online.

    const logger = winston.createLogger({
      transports: [
        new LogstashTransport({
              ...
               max_connect_retries: -1
              ...
              })]});

Run Tests

  npm test

Run integration tests with Logstash

  cd test-bench/winston-3x
  docker-compose up -d
  npm test

Inspiration

winston-loggly

License: MIT

See LICENSE for the full license text.

winston-logstash's People

Contributors

akta3d avatar cjsaylor avatar david-hodgetts avatar davidtanner avatar dazwin avatar dependabot[bot] avatar djmax avatar epd avatar gwhitelaw avatar ifraixedes avatar jaakkos avatar jeffatdeere avatar kamalaman avatar kgoerlitz avatar n2o avatar ryanbreen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

winston-logstash's Issues

Emit connection errors

winston-logstash doesn't warn me if it fails to connect to the logstash server. It would be really nice if the library emitted this event.

Logs lost on ECONNRESET

Hey,

I'm using this winston transport and am coming across an issue that's a bit of an edge case.
Essentially I have my instance setup with winston and winston-logstash and a seperate logstash server.
However, I'm experiencing issues after even a short period of being idle.
I have max_connect_retries set to -1 (unlimited).

After the idle period when I attempt to send my logs, the connection to the server results in an ECONNRESET error. However, before this point the connected boolean is set to true. The issue is that this ECONNRESET response can take as much as 30seconds to come back and during this time winston-logstash thinks it's connected so it bypasses the queue and attempts to send the logs.

When the ECONNRESET error finally arrives it behaves correctly by setting connected to false and attempting to reconnect, in turn adding all incoming logs to the queue. But again, only after the error is received back. Any logs sent during this time is essentially lost. For my API server which may not be active for a number of minutes and then a request comes in, its possible for all these logs to be lost while the connection waits to fail before re-establishing. Agreed, the TCP connection issue is my own problem, but it would be nice if winston-logstash was able to queue these logs as well in case there are problems.

I've tried a few fixes so far but no full solution just yet but some things that might be worth considering:

  • setting keep alive to true for the socket ( self.socket.setKeepAlive(true); ), or maybe as an option.
  • a method in which to add logs to the queue on failure. Current implementation is almost fire and forget.

Again, the above is an edge case but has been very troublesome.

Edit:
Working solution for me was to add self.socket.setKeepAlive(true, 60 * 1000) to line 182 of winston-logstash.js. This needs to specifically be applied after the socket is connected otherwise it doesn't seem to have any effect. This little necessity doesn't seem clear in the node.js net documentation. Anyway, hope this is useful and worth considering. Could be useful to have an options parameter.

Shouldn't throw exception when it fails to log to logstash?

My application crashes if winston-logstash fails to connect to the logstash.

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: Max retries reached, transport in silent mode, OFFLINE
    at Socket.<anonymous> (/home/hayashis/git/isdp/node_modules/winston-logstash/lib/winston-logstash.js:197:26)
    at Socket.emit (events.js:95:17)
    at TCP.close (net.js:466:12)
16 Aug 20:18:50 - [nodemon] app crashed - waiting for file changes before starting...

In my own opinion, a logging library shouldn't cause the application to crash (don't throw exception) if it fails to log... but I am not sure if the author of this library agrees with that or not?

colors are not being stripped from messages and metadata anymore

#18 seems to have been broken along the way.

require('dotenv').config();
const winston = require('winston');
const LogstashTransport = require("winston-logstash/lib/winston-logstash-latest");
const os = require('os');

// Prepare logging
const logger = winston.createLogger({
    level: "debug",
    format: winston.format.combine(winston.format.simple(), winston.format.timestamp(), winston.format.colorize()),
    defaultMeta: {
        environment: process.env.NODE_ENV || "development",
        service: 'ephemeral',
        host: os.hostname(),
        version: require('./package.json').version,
    },
    transports: [
        new winston.transports.File({ filename: 'logs/error.log', level: 'error' }),
        new winston.transports.File({ filename: 'logs/combined.log' }),
        new winston.transports.Console(),
    ],
});

if (process.env.LOGSTASH_HOST && process.env.LOGSTASH_PORT && process.env.LOGSTASH_HOST) {
    logger.info("Using Logstash");
    logger.add(        
        new LogstashTransport({
            port: process.env.LOGSTASH_PORT,
            node_name: process.env.LOGSTASH_NODE_NAME,
            host: process.env.LOGSTASH_HOST,
            max_connect_retries: -1
        }),
    );
} else {
    logger.info("Missing ENV variables to use Logstash");
}

module.exports = {
    logger: logger
}

image

Related with #6 #7 #8 issues

Hi,
It is not a new issue, it is a comment for my 3 pull requests.
I wanted to tackle in reverse order, but when I tacked it, I realised of following one (#7) and after the last one (#6).

I don't know if it was better for you make three different pull request with each issue or one with all of them, but I thought better three different to better understanding what I tried to fix.

I know if you decide to merge #6 and after the next 2, those probably will have conflicts, so let me know if you decide to merge all of them and you need help with the possible conflicts.

logs streamed together

When trying to send logs to logstash like this:

winston.info('winston info test');
winston.warn('winston warn test');
winston.error('winston error test');

I got one message in logstash:

{
  "_index": "logstash-2014.01.02",
  "_type": "logs",
  "_id": "z9TT2aR9Tq6JkkIzJRPbRw",
  "_score": null,
  "_source": {
    "message": "{\"level\":\"info\",\"message\":\"winston info test\"}\n{\"level\":\"warn\",\"message\":\"winston warn test\"}\n{\"level\":\"error\",\"message\":\"winston error test\"}\n",
    "@timestamp": "2014-01-02T11:55:46.802Z",
    "@version": "1",
    "type": "sample",
    "host": "127.0.0.1:61100"
  },
  "sort": [
    1388663746802
  ]
}

where messages are concatenated.

using codec => json_lines in tcp input won't split them apart,

but using

filter {
  split {
  }
}

will do. However I'm wondering which is the correct logstash config for logging json messages.

Module export has unpredictable signature based on Winston dependency

Hi,

I've run into a rather peculiar problem: winston-logstash exports a module signature that depends on the implicit Winston module version. This is usually not a problem, but causes code to fail when there are multiple Winston dependencies in different packages. (with NPM 3.10.8)
Here's an example:
I have 2 NPM packages in my Node project that require different versions of Winston:

  • oldPackage requires Winston 0.8.3
  • newPackage requires Winston 2.3.0 and winston-logstash

When I run NPM Install, I'll end up with the following structure:

node_modules/
  newPackage/
    node_modules/
      winston/ <-- 2.3.0
  oldPackage/
  winston/ <-- 0.8.3
  winston-logstash/

newPackage runs this code on initialization: new winston.transports.Logstash(logStashConfig);
This will throw an error: TypeError: winston.transports.Logstash is not a constructor
Debugging yields the following root cause: winston-logstash uses its own require("winston"), which, due to the lenient package dependency and aforementioned structure, will cause Winston 0.8.3 to be loaded. For newPackage, on the other hand, the loaded Winston will be 2.3.0. This causes a compatibility clash, as winston-logstash suddenly has a different Winston engine than the newPackage code that I'm trying to use in my project, which subsequently fails.

My suggestion would be, to either release a new winston-logstash version with a more recent Winston pin, ensuring that the module signature / engine is always compatible, or to let me pass the Winston engine I'd like to use into winston-logstash. Either way, one of those solutions will be necessary to avoid any version conflicts of winston-logstash's too lenient dependency pinning.

Thanks!

Invalid transport, must be an object with a log method.

Hi, I'm new to winston-logtash.
I run demo code :

var winston = require('winston');
 
  //
  // Requiring `winston-logstash` will expose
  // `winston.transports.Logstash`
  //
  require('winston-logstash');
 
  winston.add(winston.transports.Logstash, {
    port: 28777,
    node_name: 'my node name',
    host: '127.0.0.1'
  });

and I got:

Error: Invalid transport, must be an object with a log method.
    at new LegacyTransportStream (/mnt/01D5E4BA635766E0/Coding/Projects/offshorevn/offshorevn-log-analysis/node_modules/winston-transport/legacy.js:18:11)
    at DerivedLogger.add (/mnt/01D5E4BA635766E0/Coding/Projects/offshorevn/offshorevn-log-analysis/node_modules/winston/lib/winston/logger.js:345:11)
    at Object.winston.<computed> [as add] (/mnt/01D5E4BA635766E0/Coding/Projects/offshorevn/offshorevn-log-analysis/node_modules/winston/lib/winston.js:110:68)
    at Object.<anonymous> (/mnt/01D5E4BA635766E0/Coding/Projects/offshorevn/offshorevn-log-analysis/index.js:9:9)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
    at Module.load (internal/modules/cjs/loader.js:977:32)
    at Function.Module._load (internal/modules/cjs/loader.js:877:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
    at internal/main/run_main_module.js:18:47

Versions:

"winston": "^3.2.1",
 "winston-logstash": "^0.4.0"

Thanks for any helps

winston-logstash will not reconnect with logstash when using --config.reload.automatic

Hi
Thank you for this awesome repository
but I have noticed that winston-logstash will not reconnect with logstash or even throw an error when using --config.reload.automatic .
for example when I execute this command
logstash -f test.conf --config.reload.automatic
everything works well without any problem but when I try to update test.conf, winston will not print any message in console if winston.transports.Console is used or send any payload to logstash
If I tried to stop logstash and then re-execute the command above, everything will back to its normal state
this is how i call winston

const logger = winston.createLogger({
    format: winston.format.combine(
        winston.format.timestamp({
            format: 'YYYY-MM-DD HH:mm:ss',
        }),
        winston.format.printf((info) => `${info.timestamp} ${info.level}: ${info.message}${info.splat !== undefined ? `${info.splat}` : ' '}`),
    ),
    transports: [
        new logstashTransport({
            port: 28777,
            max_connect_retries: -1,
            node_name: 'DESKTOP',
            host: '127.0.0.1'}),
        new winston.transports.Console({
            format: winston.format.combine(
                winston.format.timestamp({
                    format: 'YYYY-MM-DD HH:mm:ss',
                }),
                winston.format.printf((info) => colorizer.colorize(info.level, `[${info.timestamp}] [${info.level}]: ${info.message}${info.splat !== undefined ? `${info.splat}` : ' '}`)),
            ),
        }),
        new winstonDailyRotateFile({
            filename: 'src/logs/%DATE%.log',
            datePattern: 'YYYY-MM-DD',
        })
    ],
});

do you have any idea why ?
thanks in advance

Improvement: force close method

Hi guys,

I had a problem with my application and winston-logstash.

To avoid disconnection problem, I fixed the max_connect_retries to -1 (I did this because every two minutes without data, the connection was closed by remote host).
But, when my application close properly, I need to close the logger. With the -1 set, the logger was never closed because it attempted reconnection infinitely.

So to fixe my problem, I did this:

  logger.transports.logstash.max_connect_retries = 0;
  logger.close();

It works fine but it's not very nice.

Can you add a method to force close which doesn't try to reconnect ?

Send logs with SSL to a self-signed server

I'm trying to use winston-logstash to send logs to HAProxy 1.5 which is listening on 443 and forwarding packet to a local logstash instance.

When i'm testing with with "openssl s_client -connect xxxx:443 -cert xxx.crt -key xxx.key", everything is working as long as the server certificate on HAProxy is a real one, signed by a trusted CA.

When I use a self-signed, "openssl s_client -connect xxxx:443 -cert xxx.crt -key xxx.key -CAfile xxx.pem" works, but with winston, even by adding the "ca" parameter, it's not working.

Is that a bug ?

Logstash message format

Hi.

The format of Logstash message has been changed some time ago (https://gist.github.com/jordansissel/2996677, note the message instead of @message). Sadly, these changes was not reflected in winstonjs.
Winston also has proposed change to remove logstash option in favor of custom formatter (winstonjs/winston#452). And, I think, It is a right thing to do, having logstash-related code in winston seems pretty hack-ish to me.
So I reckon that we should use custom formatter here instead of logstash : true.

Looking for maintainers

Hi Everybody!

First of all I want to thank all the contributors and users! :)

I have not been able to keep up with open issues, changes to Winston and I am not using it my self in production any more. So I feel that I not right guy to steer this project. I am looking right people to take over :)

If you are interested you can reply here or drop me email to [email protected]

thanks
Jaakko

winston.add() doesn't work?

If I add logstash transport in the constructor, I get the expected behavior (in this case, max retries reached error).
If using winston.add(), the transport fails silently...

Investigating, the differences I found were _events and _eventsCount:

// [email protected]
// [email protected]
// [email protected]

'use strict'

const winston = require('winston')
require('winston-logstash')

const logger1 = new winston.Logger({
  transports: [
    new winston.transports.Logstash({
      host: '127.0.0.1',
      port: '8200',
      node_name: 'node_logger',
    }),
  ],
})

const logger2 = new winston.Logger()
logger2.add(winston.transports.Logstash, {
  host: '127.0.0.1',
  port: '8200',
  node_name: 'node_logger',
})

console.log(logger1.transports.logstash._events) // logs {}
console.log(logger2.transports.logstash._events) // logs { error: [Function: bound ] }

Am I doing something wrong? It'd be nice to use .add() in my case...

Thanks!

Trouble parsing json

Message sent from winston-logstash is not parsable and appararently to short.

In logstash.log I see

{:timestamp=>"2015-06-23T14:19:15.934000+0200", :message=>"Received an event that has a different character encoding than you configured.", :text=>"\\xFF\\xF4\\xFF\\xFD\\u0006\\r", :expected_charset=>"UTF-8", :level=>:warn}
{:timestamp=>"2015-06-23T14:19:15.937000+0200", :message=>"Trouble parsing json", :source=>"message", :raw=>"\\xFF\\xF4\\xFF\\xFD\\u0006\\r", :exception=>#<JSON::ParserError: unexpected token at '\xFF\xF4\xFF\xFD\u0006\r'>, :level=>:warn}

Same thing in kibana / elastic search.
https://www.dropbox.com/s/y8shba4tj88z8ac/Screenshot%202015-06-23%2014.28.25.png?dl=0

My logstash.conf:

input {
  tcp {
    port => 28777
    type => "winston"
  }
  relp {
    port => 2514
    type => "syslog"
    debug => true
  }
}

filter {
  if [type] == "winston" {
    json {
      source => "message"
    }
  }

  if [type] == "syslog" {

    # strip the syslog PRI part and create facility and severity fields.
    # the original syslog message is saved in field %{syslog_raw_message}.
    # the extracted PRI is available in the %{syslog_pri} field.
    #
    # You get %{syslog_facility_code} and %{syslog_severity_code} fields.
    # You also get %{syslog_facility} and %{syslog_severity} fields if the
    # use_labels option is set True (the default) on syslog_pri filter.
    grok {
        match => {
          "message" => [ "<%{POSINT:syslog_pri}>%{SPACE}%{GREEDYDATA:message}" ]
        }
        add_tag => "got_syslog_pri"
        add_field => [ "syslog_raw_message", "%{message}" ]
        overwrite => "message"
    }

    if "got_syslog_pri" in [tags] {
      syslog_pri { }
    }

    grok {
        match => {
          "message" => ["%{SYSLOGLINE}", "%{PWMSYSLOGLINE}"]
        }
        add_field => [ "received_at", "%{@timestamp}" ]
        add_field => [ "received_from", "%{logsource}" ]
        overwrite => "message"
    }

    date {
        match => ["timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss"]
        remove_field => ["timestamp"]
    }

    date {
        match => ["timestamp8601", "ISO8601"]
        remove_field => ["timestamp8601"]
    }

    if [program] == "PWM" {
      json {
        source => "message"
        target => "data"
      }
    }

    if [program] == "nginx" {
      if [syslog_severity] == "informational" {
        grok {
          match => { "message" => "%{NGINXACCESS}" }
        }
      } else {
        grok {
          match => {
            "message" => ["%{NGINXERRORDETAIL}", "%{NGINXERROR}"]
          }
          overwrite => "message"
        }
      }
      mutate {
        add_field => { "server_name" => "%{server}" }
      }
      mutate {
        gsub => [ "server_name", "\.", "_"]
      }
      if [clientip] {
        mutate {
          add_field => { "clientfqdn" => "%{clientip}" }
        }
        dns {
          reverse => [ "clientfqdn" ] action => "replace"
        }
        geoip {
          source => "clientip"
          database => "/opt/geo/GeoLiteCity.dat"
        }
      }
    }
  }
}

output {
  if [program] == "nginx" {
    if [syslog_severity] == "informational" {
      statsd {
        host => "localhost"
        port => 8125
        sender => "%{logsource}"
        # Count one hit every event by response
        increment => [ "nginx.%{server_name}.response.%{response}" ]
        # Use the 'bytes' field from the apache log as the count value.
        count => [ "nginx.%{server_name}.bytes", "%{bytes}" ]
      }
    }
  }
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

My box: Ubuntu 12.04 LTS
logstash version: 1.4.2
elastic search version: 1.0.3

Any suggestions? Thanks.

deprecated setting in recommended config

+1 for open-sourcing this :)

There's a small something that maybe the TODO list already covers. Putting the recommended configuration into a file for logstash to run:

# from https://github.com/jaakkos/winston-logstash
input {
   # Sample input over TCP
   tcp { format => "json" charset => "UTF-8" port => 28777 type=>"sample" }
}
output {
   stdout { debug => true debug_format => "json"}
}

triggers logstash deprecation warnings, the first of which being:
You are using a deprecated config setting "format" set in tcp. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. You should use the newer 'codec' setting instead.

Would be nice updating to the new format, given this module requires specific logstash configuration.

Can you publish in npm?

Hi Jaakkos,

I appreciate your fast response.
I would like to ask you one favour more, when you have a chance, can you update the npm registry?

Thanks so much for your time.

unsafe stringify and JSON circular structure

Hi,
if I pass an axios error object to the transport's .error() function, the logger crashes with the following exception:

2024-05-05 09:59:11.394  Error: app logged out: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'httpAgent' -> object with constructor 'Agent'
    |     property 'sockets' -> object with constructor 'Object'
    |     ...
    |     property 'errored' -> object with constructor 'Object'
    --- property 'config' closes the circle
2024-05-05 09:59:11.397  TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'httpAgent' -> object with constructor 'Agent'
    |     property 'sockets' -> object with constructor 'Object'
    |     ...
    |     property 'errored' -> object with constructor 'Object'
    --- property 'config' closes the circle
    at JSON.stringify (<anonymous>)
    at LogstashTransport.log (/app/node_modules/.pnpm/[email protected][email protected]/node_modules/winston-logstash/lib/winston-logstash-latest.js:34:27)
    at LogstashTransport._write (/app/node_modules/.pnpm/[email protected]/node_modules/winston-transport/index.js:82:19)
    at doWrite (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_writable.js:390:139)
    at writeOrBuffer (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_writable.js:381:5)
    at Writable.write (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_writable.js:302:11)
    at DerivedLogger.ondata (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_readable.js:629:20)
    at DerivedLogger.emit (node:events:529:35)
    at addChunk (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_readable.js:279:12)
    at readableAddChunk (/app/node_modules/.pnpm/[email protected]/node_modules/readable-stream/lib/_stream_readable.js:262:11)

a related issue is: winstonjs/winston#1946
where it was suggested that fast-safe-stringify should be used instead of JSON.stringify().
in the code I see several refernces to stringify libs, but they are all in the test bench.

Cannot get to work with winston 3

Does this even work with winston 3?

I get the following error when using the latest version of winston.

VSCODE Error Image

These are my dependencies in my package.json

"dependencies": {
"body-parser": "^1.18.2",
"firebase": "^4.6.2",
"firebase-admin": "^5.5.1",
"message-client": "^1.0.5",
"sails-disk": "^0.10.10",
"sails-memory": "^0.10.7",
"sails-postgresql": "^0.11.4",
"waterline": "^0.12.2",
"winston": "^3.0.0-rc1",
"winston-logstash": "^0.4.0"
}

object appears in logstash as a string rather than being included as a json subobject

Passing an object to the logger, logstash shows the object as a string of a json object, rather than adding the object as a json subobject of the logstash entry. Using the json codec. E.g.

{
  "_index": "logstash-2014.01.17",
  "_type": "zwinston-logstash",
  "_id": "gZYydBqYQr2NJTwKh0GplQ",
  "_score": null,
  "_source": {
    "message": "{\"metaType\":\"regular\",\"text\":\"“Minorities”\",\"stylesArray\":[[\"l\"]],\"finalStyles\":{},\"positionInfo\":{},\"level\":\"warn\",\"message\":\"\"}\n{\"level\":\"warn\",\"message\":\"No final styles applied to token\"}\n{\"metaType\":\"delimiter\",\"stylesArray\":[[\"l\"]],\"finalStyles\":{},\"positionInfo\":{},\"level\":\"warn\",\"message\":\"\"}\n{\"level\":\"warn\",\"message\":\"No final styles applied to token\"}\n{\"metaType\":\"regular\",\"text\":\"patterns\",\"stylesArray\":[[\"l\"]],\"finalStyles\":{},\"positionInfo\":{},\"level\":\"warn\",\"message\":\"\"}\n{\"level\":\"warn\",\"message\":\"No final styles applied to token\"}\n{\"metaType\":\"delimiter\",\"stylesArray\":[[\"l\"]],\"finalStyles\":{},\"positionInfo\":{},\"level\":\"warn\",\"message\":\"\"}\n{\"level\":\"warn\",\"message\":\"No final styles applied to token\"}\n{\"metaType\":\"regular\",\"text\":\"comparison\",\"stylesArray\":[[\"l\"]],\"finalStyles\":{},\"positionInfo\":{},\"level\":\"warn\",\"message\":\"\"}\n",
    "@version": "1",
    "@timestamp": "2014-01-17T19:32:33.822Z",
    "host": "xxxxxxxxxx:yyyyy",
    "type": "zwinston-logstash"
  },
  "sort": [
    1389987153822,
    1389987153822
  ]
}

Is this something that can be avoided from winston-logstash's side?
It consistently happens also when using:

filter {
  split { }
  json { source => "message" }
}

The object may contain circular references, but that typically gets safely handled in similar circumstances; I also see a test for handling circular references included in this repo so I assume this is not the cause.

How to change timestamp format.

How to change "timestamp":"2017-02-07T12:25:32.853Z" to something like "timestamp":"yyyy-MM-dd HH:mm:ss"
log-example -
{"level":"info","message":"Hello winston","timestamp":"2017-02-07T12:25:32.853Z"}

winston-logstash + npm = <3

Hey,

Any plan to add this to npm? Would be a delight rather than having the github dependency in my package.json.

Cheers.

Test are broken in Travis with node 8 issue with error "Uncaught Error: socket hang up"

> [email protected] test /home/travis/build/jaakkos/winston-logstash
> node_modules/mocha/bin/mocha test/*_test.js
  winston-logstash transport
    with logstash server
      ✓ send logs over TCP as valid json
      ✓ send each log with a new line character
      ✓ send with different log levels
      ✓ send with overrided meta data
    with secured logstash server
      ✓ send logs over SSL secured TCP as valid json
      1) send logs over SSL secured TCP as valid json with SSL verification
      ✓ logstash transport receive an error when there is a connection error different from ECONNREFUSED
    without logstash server
      ✓ fallback to silent mode if logstash server is down (403ms)
      ✓ emit an error message when it fallback to silent mode (403ms)
  8 passing (864ms)
  1 failing
  1) winston-logstash transport
       with secured logstash server
         send logs over SSL secured TCP as valid json with SSL verification:
     Uncaught Error: socket hang up
      at TLSSocket.onHangUp (_tls_wrap.js:1135:19)
      at endReadableNT (_stream_readable.js:1056:12)
      at _combinedTickCallback (internal/process/next_tick.js:138:11)
      at process._tickCallback (internal/process/next_tick.js:180:9)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.