This library is provided to allow standard python logging to output log data as json objects ready to be shipped out to logstash.
Pip (PyPI):
pip install logstash_formatter
Manual:
python setup.py install
Json outputs are provided by the LogstashFormatter logging formatter.
import logging from logstash_formatter import LogstashFormatterV1 logger = logging.getLogger() handler = logging.StreamHandler() formatter = LogstashFormatterV1() handler.setFormatter(formatter) logger.addHandler(handler)
The LogstashFormatter may take the following named parameters:
fmt
: Config as a JSON string that supports:extra
: provide extra fields always present in logs.source_host
: override source host name.
json_cls
: JSON encoder to forward tojson.dump
.json_default
: Default JSON representation for unknown types, by default coerce everything to a string.
LogstashFormatterV1
adheres to the more 1.2.0 schema and will not update
fields, apart from a special handling of msg
which will be updated to
message
when applicable.
You can also add extra fields to your json output by specifying a dict in place of message, or by specifying
the named argument extra
as a dictionary. When supplying the exc_info
named argument with a truthy value,
and if an exception is found on the stack, its traceback will be attached to the payload as well.
logger.info({"account": 123, "ip": "172.20.19.18"}) logger.info("classic message for account: {account}", extra={"account": account}) try: h = {} h['key'] except: logger.info("something unexpected happened", exc_info=True)
The following keys will be found in the output JSON:
@source_host
: source hostname for the log@timestamp
: ISO 8601 timestamp@message
: short message for this log@fields
: all extra fields
{ "@fields": { "account": "pyr", "args": [], "created": 1367480388.013037, "exception": [ "Traceback (most recent call last):\n", " File \"test.py\", line 16, in <module>\n k['unknown']\n", "KeyError: 'unknown'\n" ], "filename": "test.py", "funcName": "<module>", "levelname": "WARNING", "levelno": 30, "lineno": 18, "module": "test", "msecs": 13.036966323852539, "name": "root", "pathname": "test.py", "process": 1819, "processName": "MainProcess", "relativeCreated": 18.002986907958984, "thread": 140060726359808, "threadName": "MainThread" }, "@message": "TEST", "@source_host": "phoenix.spootnik.org", "@timestamp": "2013-05-02T09:39:48.013158" }
The following keys will be found in the output JSON:
@timestamp
: ISO 8601 timestamp@version
: Version of the schema
{"@version": 1, "account": "pyr", "lineno": 1, "levelno": 30, "filename": "test.py", "thread": 140566036444928, "@timestamp": "2015-03-30T09:46:23.000Z", "threadName": "MainThread", "relativeCreated": 51079.52117919922, "process": 10787, "source_host": "phoenix.spootnik.org", "processName": "MainProcess", "pathname": "test.py", "args": [], "module": "test", "msecs": 999.9005794525146, "created": 1427708782.9999006, "name": "root", "stack_info": null, "funcName": "<module>", "levelname": "WARNING", "message": "foo"}