Giter VIP home page Giter VIP logo

logxi's Introduction

demo

logxi

log XI is a structured 12-factor app logger built for speed and happy development.

  • Simpler. Sane no-configuration defaults out of the box.
  • Faster. See benchmarks vs logrus and log15.
  • Structured. Key-value pairs are enforced. Logs JSON in production.
  • Configurable. Enable/disalbe Loggers and levels via env vars.
  • Friendlier. Happy, colorful and developer friendly logger in terminal.
  • Helpul. Traces, warnings and errors are emphasized with file, line number and callstack.
  • Efficient. Has level guards to avoid cost of building complex arguments.

Requirements

Go 1.3+

Installation

go get -u github.com/mgutz/logxi/v1

Getting Started

import "github.com/mgutz/logxi/v1"

// create package variable for Logger interface
var logger log.Logger

func main() {
    // use default logger
    who := "mario"
    log.Info("Hello", "who", who)

    // create a logger with a unique identifier which
    // can be enabled from environment variables
    logger = log.New("pkg")

    // specify a writer, use NewConcurrentWriter if it is not concurrent
    // safe
    modelLogger = log.NewLogger(log.NewConcurrentWriter(os.Stdout), "models")

    db, err := sql.Open("postgres", "dbname=testdb")
    if err != nil {
        modelLogger.Error("Could not open database", "err", err)
    }

    fruit := "apple"
    languages := []string{"go", "javascript"}
    if log.IsDebug() {
        // use key-value pairs after message
        logger.Debug("OK", "fruit", fruit, "languages", languages)
    }
}

logxi defaults to showing warnings and above. To view all logs

LOGXI=* go run main.go

Highlights

This logger package

  • Is fast in production environment

    A logger should be efficient and minimize performance tax. logxi encodes JSON 2X faster than logrus and log15 with primitive types. When diagnosing a problem in production, troubleshooting often means enabling small trace data in Debug and Info statements for some period of time.

    # primitive types
    BenchmarkLogxi          100000    20021 ns/op   2477 B/op    66 allocs/op
    BenchmarkLogrus          30000    46372 ns/op   8991 B/op   196 allocs/op
    BenchmarkLog15           20000    62974 ns/op   9244 B/op   236 allocs/op
    
    # nested object
    BenchmarkLogxiComplex    30000    44448 ns/op   6416 B/op   190 allocs/op
    BenchmarkLogrusComplex   20000    65006 ns/op  12231 B/op   278 allocs/op
    BenchmarkLog15Complex    20000    92880 ns/op  13172 B/op   311 allocs/op
    
  • Is developer friendly in the terminal. The HappyDevFormatter is colorful, prints file and line numbers for traces, warnings and errors. Arguments are printed in the order they are coded. Errors print the call stack.

    HappyDevFormatter is not too concerned with performance and delegates to JSONFormatter internally.

  • Logs machine parsable output in production environments. The default formatter for non terminals is JSONFormatter.

    TextFormatter may also be used which is MUCH faster than JSON but there is no guarantee it can be easily parsed.

  • Has level guards to avoid the cost of building arguments. Get in the habit of using guards.

    if log.IsDebug() {
        log.Debug("some ", "key1", expensive())
    }
    
  • Conforms to a logging interface so it can be replaced.

    type Logger interface {
        Trace(msg string, args ...interface{})
        Debug(msg string, args ...interface{})
        Info(msg string, args ...interface{})
        Warn(msg string, args ...interface{}) error
        Error(msg string, args ...interface{}) error
        Fatal(msg string, args ...interface{})
        Log(level int, msg string, args []interface{})
    
        SetLevel(int)
        IsTrace() bool
        IsDebug() bool
        IsInfo() bool
        IsWarn() bool
        // Error, Fatal not needed, those SHOULD always be logged
    }
    
  • Standardizes on key-value pair argument sequence

log.Debug("inside Fn()", "key1", value1, "key2", value2)

// instead of this log.WithFields(logrus.Fields{"m": "pkg", "key1": value1, "key2": value2}).Debug("inside fn()")

    logxi logs `FIX_IMBALANCED_PAIRS =>` if key-value pairs are imbalanced

    `log.Warn and log.Error` are special cases and return error:

    ```go
return log.Error(msg)               //=> fmt.Errorf(msg)
return log.Error(msg, "err", err)   //=> err
  • Supports Color Schemes (256 colors)

    log.New creates a logger that supports color schemes

    logger := log.New("mylog")
    

    To customize scheme

    # emphasize errors with white text on red background
    LOGXI_COLORS="ERR=white:red" yourapp
    
    # emphasize errors with pink = 200 on 256 colors table
    LOGXI_COLORS="ERR=200" yourapp
    
  • Is suppressable in unit tests

func TestErrNotFound() { log.Suppress(true) defer log.Suppress(false) ... }




## Configuration

### Enabling/Disabling Loggers

By default logxi logs entries whose level is `LevelWarn` or above when
using a terminal. For non-terminals, entries with level `LevelError` and
above are logged.

To quickly see all entries use short form

    # enable all, disable log named foo
    LOGXI=*,-foo yourapp

To better control logs in production, use long form which allows
for granular control of levels

    # the above statement is equivalent to this
    LOGXI=*=DBG,foo=OFF yourapp

`DBG` should obviously not be used in production unless for
troubleshooting. See `LevelAtoi` in `logger.go` for values.
For example, there is a problem in the data access layer
in production.

    # Set all to Error and set data related packages to Debug
    LOGXI=*=ERR,models=DBG,dat*=DBG,api=DBG yourapp

### Format

The format may be set via `LOGXI_FORMAT` environment
variable. Valid values are `"happy", "text", "JSON", "LTSV"`

    # Use JSON in production with custom time
    LOGXI_FORMAT=JSON,t=2006-01-02T15:04:05.000000-0700 yourapp

The "happy" formatter has more options

*   pretty - puts each key-value pair indented on its own line

    "happy" default to fitting key-value pair onto the same line. If
    result characters are longer than `maxcol` then the pair will be
    put on the next line and indented

*   maxcol - maximum number of columns before forcing a key to be on its
    own line. If you want everything on a single line, set this to high
    value like 1000. Default is 80.

*   context - the number of context lines to print on source. Set to -1
    to see only file:lineno. Default is 2.


### Color Schemes

The color scheme may be set with `LOGXI_COLORS` environment variable. For
example, the default dark scheme is emulated like this

    # on non-Windows, see Windows support below
    export LOGXI_COLORS=key=cyan+h,value,misc=blue+h,source=magenta,TRC,DBG,WRN=yellow,INF=green,ERR=red+h
    yourapp

    # color only errors
    LOGXI_COLORS=ERR=red yourapp

See [ansi](http://github.com/mgutz/ansi) package for styling. An empty
value, like "value" and "DBG" above means use default foreground and
background on terminal.

Keys

*   \*  - default color
*   TRC - trace color
*   DBG - debug color
*   WRN - warn color
*   INF - info color
*   ERR - error color
*   message - message color
*   key - key color
*   value - value color unless WRN or ERR
*   misc - time and log name color
*   source - source context color (excluding error line)

#### Windows

Use [ConEmu-Maximus5](https://github.com/Maximus5/ConEmu).
Read this page about [256 colors](https://code.google.com/p/conemu-maximus5/wiki/Xterm256Colors).

Colors in PowerShell and Command Prompt _work_ but not very pretty.

## Extending

What about hooks? There are least two ways to do this

*   Implement your own `io.Writer` to write to external services. Be sure to set
    the formatter to JSON to faciliate decoding with Go's built-in streaming
    decoder.
*   Create an external filter. See `v1/cmd/filter` as an example.

What about log rotation? 12 factor apps only concern themselves with
STDOUT. Use shell redirection operators to write to a file.

There are many utilities to rotate logs which accept STDIN as input. They can
do many things like send alerts, etc. The two obvious choices are Apache's `rotatelogs`
utility and `lograte`.

```sh
yourapp | rotatelogs yourapp 86400

Testing

# install godo task runner
go get -u gopkg.in/godo.v2/cmd/godo

# install dependencies
godo install -v

# run test
godo test

# run bench with allocs (requires manual cleanup of output)
godo bench-allocs

License

MIT License

logxi's People

Contributors

flyingmutant avatar ian-kent avatar karlmutch avatar xboston avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

logxi's Issues

Feature: option to honor Stringer interface

If I have a large struct, logxi's map rendering of it can be too verbose. I can implement String() on it, but logxi ignores it. If I pass s.String() as the arg to logxi, I incur the Sprintf cost even when the log message is ignored, so I need to do this:

if log.isTrace() { log.Trace("the struct", s.String())}

Would be cool if logxi had a flag to enable using String() if implemented.

Error Installing

When i try to install the package i get this error:

> go get -u github.com/mgutz/logxi/v1
# github.com/mgutz/logxi/v1
../../mgutz/logxi/v1/callstack.go:11: import /home/deploy/golang/pkg/linux_amd64/github.com/mgutz/ansi.a: not a package file

Additional Info:

OS: Debian 7.1

> go version
go version go1.4.2 linux/amd64
> go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/deploy/golang"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"

Safe concurrent logging

As far as I was able to check the internals of the package, there's no guarding mechanism when writting to stdout.

If this is correct, then the library is not safe to use with concurrent code:

The standard log package does make use of locking mechanisms to safe-guard against this.

Maintainers?

Are you interested in getting help maintaining logxi? I'd be happy to help with the outstanding issues if you are open to letting other people help you. There is little activity here and I'd preferably not fork the original project.

@mgutz

Why does Warn and Error return the error?

Im failing to understand the purpose of returning back the error i sent to log.Error and log.Warn. I already have the error and the source code shows that you do not change anything about the error. What's the deal with this behaviour? I can't find any good documentation on this.

Warn should return error object

Hey first of all I absolutely love logxi & feel bad about posting this first "issue" that is really a feature request!

Request: can we have Warn() return the first error param like Error() does.

Reasoning by real example:-

I have a function that unpacks a packet received from the network, and analyzes its fields to make sure they are all legal in relation to the protocol and state transitions they will drive. In some circumstances, the packet will be illegal in the sense that it breaks a sequence of transitions, or something like that, and thus the function returns a predefined error describing what has happened, and higher level code resets the protocol. In this case the protocol is an overlay over and existing P2P network and such conditions are triggered quite naturally by various things and so resetting the virtual connection is quite normal and only merits a warning in the logs. Nonetheless, I still want to pass up the error so the protocol can be reset.

Would there be any probs with having Warn return errors in the same way?

Best, Dominic

Option to disable colors

When running under LiteIDE the colors make it really hard to read the program output (the LiteIDE terminal does not handle the ANSI escape sequences, as such, we see them intermingled with the actual program output).

Can we have an option to disable the colors? maybe LOGXI_COLORS=disable

Changing the defaults

I've been using logxi for a while. One of the things that continues to itch is that the defaults are very hard to control.

logxi establishes it's settings by reading environment variables in a package init() function. All loggers created after that pick up those settings. It's not possible to reliable modify those environment variables or change logxi's built-in defaults before logxi's init() function is executed.

What we're doing is running logxi's ProcessLogix*Env() functions, then recreating all our loggers to pick up the new settings, but that has a big drawback. It means there needs to be a single block of code that knows about all the loggers that need to be recreated. That discourages the use of package-private loggers.

I'm not sure what or if there's a good solution. I see there's a logger map protected by a mutex with a comment about future support for runtime configuration changes...but since most code has direct references to logger instances...there would need to be an additional, concurrent-safe indirection between logger references and logger configurations that could change.

For now, I'm considering just forking logxi and added the bootstrap code to load environment defaults before logxi's init sequence.

Configuration problem.

I need to run my app with LOGXI and LOGXI_FORMAT and

LOGXI=*,LOGXI_FORMAT=JSON ./myapp

not working.

Support context logger

would be awesome if this library would support context loggers, so you can for example at each request set variables in the logger and pass it along to other parts of the code so at every log call would have a request specific number.

Time more precise by default.

Hello,
would it be possible to make default "_t" format more precise than seconds? Ideally ms or us.

The reason for this is following:
If you have some log parsing tool that puts data say in ElasticSearch for later processing, and you want to filter this data by time of something happening, one second is really really coarse grained. A LOT of events will happen in one second, and we've had problems with this in other applications that log with low precision (seconds being most precise).

I don't think this would slow down the logging process or make the logfiles too big, I believe this will have mostly benefits.

Thank you.

go modules build fails

Under a project that depends on github.com/mgutz/logxi, when I migrate the dependency management to go mod, and run go build, then I see:

go: finding github.com/mgutz/logxi latest
build github.com/myorg/myproj/cmd: cannot load github.com/mgutz/logxi: cannot find module providing package github.com/mgutz/logxi

It's probably because there's no source file in the top directory, but only under v1.

See also a related issue: influxdata/influxdb1-client#5

Log as csv

Wondering if there is a CSV formatter ?
Needed for business analytics guys as they love to suck up the CSV into their charting software :)

Call stack is strange and shows up even on .Info() when passing an error variable.

Is this expected?

❯❯❯ GOMAXPROCS=4 LOGXI="*=INF" ./foo
17:59:21.391877 INF foo Error on parsing message (continuing)
   err: invalid character 's' looking for beginning of value
   /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:48 (0x4b58bf)
        (*JSONFormatter).writeError: jf.set(buf, KeyMap.CallStack, string(debug.Stack()))
/home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:87 (0x4b5c4f)
        (*JSONFormatter).appendValue: jf.writeError(buf, err)
/home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:114 (0x4b643c)
        (*JSONFormatter).set: jf.appendValue(buf, val)
/home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:166 (0x4b6c36)
        (*JSONFormatter).Format: jf.set(buf, key, args[i+1])
/home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:187 (0x4b6ffc)
        (*JSONFormatter).LogEntry: jf.Format(buf, level, msg, args)
/home/francisco/.go/src/github.com/mgutz/logxi/v1/happyDevFormatter.go:273 (0x4b2b9b)
        (*HappyDevFormatter).Format: entry := hd.jsonFormatter.LogEntry(level, msg, args)
/home/francisco/.go/src/github.com/mgutz/logxi/v1/defaultLogger.go:106 (0x4aed00)
        (*DefaultLogger).Log: l.formatter.Format(l.writer, level, msg, args)
/home/francisco/.go/src/github.com/mgutz/logxi/v1/defaultLogger.go:70 (0x4ae8c3)
        (*DefaultLoggelogger.Info("Error on parsing message (continuing)", "err", err)r).Info: l.Log(LevelInfo, msg, args)
/home/francisco/.go/src/github.com/foo/main.go:158 (0x402efe)
        main: logger.Info("Error on parsing message (continuing)", "err", err)
/usr/local/go/src/runtime/proc.go:63 (0x4173a3)
        main: main_main()
/usr/local/go/src/runtime/asm_amd64.s:2232 (0x42a411)
        goexit:

This gets generated at logger.Info("Error on parsing message (continuing)", "err", err).

If I do logger.Error("Error on parsing message (continuing)", "err", err.Error()) instead I get:

❯❯❯ ./timelapsee
18:07:11.060864 ERR foo Error on parsing message (continuing)
   err: invalid character 'a' looking for beginning of value
/home/francisco/.go/src/github.com/foo/main.go:158 (0x402f31)
        main: logger.Error("Error on parsing message (continuing)", "err", err.Error())
/usr/local/go/src/runtime/proc.go:63 (0x4173d3)
        main: main_main()
/usr/local/go/src/runtime/asm_amd64.s:2232 (0x42a441)
        goexit: 

Which is the expected call stack for .Error, if I use .Info instead I get no call stack as expected.

The docs at the README show samples passing error variables directly, but this seems to generate some unwanted behavior.

Push to s3

This looks really nice.
I need to a hook to log to s3. Is this possible.

Fit formatter

Hello. There 3 formatters.

RegisterFormatFactory(FormatHappy, formatFactory)
    RegisterFormatFactory(FormatText, formatFactory)
    RegisterFormatFactory(FormatJSON, formatFactory)

So the question is - what "fit" means.
You're parsing env, there are not fit there. So it assignes formatterFormat to "fit", (and overrides previous value, happy e.g.).

    for key, value := range m {
        switch key {
        default:
            formatterFormat = key
        case "t":
            tFormat = value
        case "pretty":
            isPretty = value != "false" && value != "0"
        case "maxcol":
            col, err := strconv.Atoi(value)
            if err == nil {
                maxCol = col
            } else {
                maxCol = defaultMaxCol
            }
        case "context":
            lines, err := strconv.Atoi(value)
            if err == nil {
                contextLines = lines
            } else {
                contextLines = defaultContextLines
            }
        case "LTSV":
            formatterFormat = "text"
            AssignmentChar = ltsvAssignmentChar
            Separator = ltsvSeparator
        }
    }

you have it on
main.go

p.Task("isolate", do.S{"build"}, func(c *do.Context) {
        c.Bash("LOGXI=* LOGXI_FORMAT=fit,maxcol=80,t=04:05.000,context=2 demo", do.M{"$in": "v1/cmd/demo"})
    })

init.go

        defaultLogxiFormatEnv = "happy,fit,maxcol=80,t=15:04:05.000000,context=-1"

i think it should be removed?

Consider documenting requirements

Yesterday I've tried it on latest Ubuntu (14.10) with the default Go for the distribution (1.2.X) and it didn't compile. It seems to require 1.3 (it would be nice to have 1.2 supported by the way).

Text formatter formats Trace incorrectly

When using the text formatter, the following two lines show the difference between debug and trace:

_t: 08:55:55.248623 _p: 22564 _n: vault _l: DBG _m: core: cluster name set name: vault-cluster-d1579c8d
_t: 08:55:55.248631core: cluster ID not found, generating new

Not only is it lacking various information but there is also no space between the timestamp and the message.

Is this on purpose? If not I'll take a stab at fixing it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.