humanlogio / humanlog Goto Github PK

I'm working to make better use of structured logging, and came across your humanlog utility.
Operating System: OSX
Terminal Client: Bash/Default

I have a small test log I created using gokit.
{"caller":"main.go:25","level":"debug","msg":"Testing Debug","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:26","level":"info","msg":"Testing Info","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:27","level":"warn","msg":"Testing Warn","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:28","level":"error","msg":"Testing Error","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:29","level":"crit","msg":"Testing Crit","service":"logtestapp","ts":"2016-07-21T19:50:18Z"}

I installed humanlog using the go get method from your README.
go get -u github.com/aybabtme/humanlog/...

Once installed, I run the following command, but am not seeing any change to display of the logs I'm providing, save for the humanlog> reading stdin... at the top of the ouput.
humanlog < logtestapp.log

Is there something I'm missing, or a potential misconfiguration I've encountered? Anything that can point me in the right direction would be greatly appreciated, thanks!

From the main README.md I got the impression that humanlog can handle arbitrary logfmt data ("If you emit logs in JSON or in logfmt, you will enjoy pretty logs..."), and then I wondered why it left some simple logs completely unchanged, like:

msg=hello

After diving into the source I found that a log entry must have a level, time, and msg field (https://github.com/aybabtme/humanlog/blob/master/logrus_handler.go#L39). It would be useful to mention this.

About the implementation:

• "lvl" and "message" seem to be treated as aliases, but the sanity check doesn't include them, so I'm not sure whether those aliases really work?
• the check expects "msg" to be quoted. This is too strict, for example the go-logfmt/logfmt Encoder doesn't put quotation marks around values unless absolutely necessary.

Great library.
Minor edit to README:
go install github.com/aybabtme/humanlog/cmd/humanlog@latest

Consider the following line:

time="2019-12-09T13:56:00Z" level=trace msg="crash" somekey=

This makes humanlog crash horribly:

[root@la1 ~]# ./humanlog < repro.log
humanlog> reading stdin...
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/aybabtme/humanlog/parser/logfmt.scanAllKeyValue(0xc42009e000, 0x3c, 0x1000, 0x101, 0xc420045ab0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/parser/logfmt/logfmt_parser.go:38 +0x31e
github.com/aybabtme/humanlog/parser/logfmt.Parse(0xc42009e000, 0x3c, 0x1000, 0x101, 0xc420045ab0, 0x20)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/parser/logfmt/logfmt_parser.go:18 +0x5b
github.com/aybabtme/humanlog.Scanner(0x5c64e0, 0xc42000c018, 0x5c6500, 0xc42000c020, 0x690ca0, 0x2, 0x2)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/scanner.go:43 +0x405
main.newApp.func1(0xc420082160, 0xc420082160, 0xc420045ce7)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/cmd/humanlog/main.go:124 +0x369
github.com/aybabtme/humanlog/vendor/github.com/urfave/cli.HandleAction(0x56f620, 0xc420098000, 0xc420082160, 0xc4200522a0, 0x0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/vendor/github.com/urfave/cli/app.go:501 +0xc8
github.com/aybabtme/humanlog/vendor/github.com/urfave/cli.(*App).Run(0xc420094000, 0xc42000e1b0, 0x1, 0x1, 0x0, 0x0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/vendor/github.com/urfave/cli/app.go:268 +0x60c
main.main()
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/cmd/humanlog/main.go:31 +0x11c

This is evidently caused by the somekey= having a zero length value

Despite calling humanlog with --light-bg, additional values get colored with a very light color that is almost unreadable on a white background.

Apparently because of:

ValColor:          color.New(color.FgHiWhite),

Should different colors be used depending on --light-bg?

The default colors don't work for me:

Due to the way that the underlying rgbterm works, I can't just send a quick \033[40m to set the background color to dark; it only works for the first colored block printed, and then a SGR reset kills that.

It's possible to set the default background color, but it's very terminal dependent, and so my wrapper function for Scanner is getting really ugly:

func ScannerDarkly(src io.Reader, dst io.Writer, opts *humanlog.HandlerOptions) {
    var setBgColorCode string
    if strings.Contains(os.Getenv("TERM_PROGRAM"), "iTerm") {
        // iTerm...
        setBgColorCode = "\033]Ph%.2x%.2x%.2x\033\\"
    } else if strings.Contains(os.Getenv("TERM"), "256") || strings.Contains(os.Getenv("TERM"), "xterm") {
        setBgColorCode = "\033]6;rgb:%.2x/%.2x/%.2x\a"
    }
    if setBgColorCode != "" {
        dst.Write([]byte(fmt.Sprintf(setBgColorCode, bgColor[0], bgColor[1], bgColor[2])))
    }
    humanlog.Scanner(src, dst, opts)
}

I'm not cleaning up the color change, either, so when my program quits it leaves the background dark. Do you think it would be possible to make the code accept some kind of color palette configuration or something (perhaps with regular ANSI code fallbacks), instead of all of the hardcoded color codes?

Also don't log update check errors when there's no internet. Stuff like that.

Would be awesome to have this in the Arch User Repository as well :)

https://wiki.archlinux.org/title/Arch_User_Repository

$ echo '{"level":"info", "time" : 1533094882, "msg":"logger construction succeeded"}' | humanlog

I get:

humanlog> reading stdin...
{"level":"info", "time" : 1533094882, "msg":"logger construction succeeded"}

And nothing else is happening. I'm hot sure i've built the project correctly, cd

$ cmd/humanlog && go build got me a seemingly working binary

I would like to be able to add a custom columns from specific fields, one use case example:

The logs I am using (from python-json-logger" have a field called "name" that shows what module generated that log message
The way it looks now:

https://github.com/madzak/python-json-logger and other python json loggers tend to use asctime field.

This reflects the fieldname in python stdlib logging framework https://docs.python.org/3/library/logging.html

Would be great if asctime was automatically supported.

I followed given instructions but not working on my mac.

Hi,

I have a long json payload in logfmt value, and I run into token too long.
Could you please increase the buffer, make it configurable, and perhaps at least catch the particular error and instead of crashing, mark the field as error and continue?

I found some background on the error: https://stackoverflow.com/a/37455465

When I throw some log at humanlog, it fails with a runtime panic in the parser.

I can't share the log and before I try to narrow it down further let me first ask whether it would make more sense to replace the builtin parser with https://github.com/go-logfmt/logfmt?

Warning: Calling bottle :unneeded is deprecated! There is no replacement.
Please report this issue to the aybabtme/tap tap (not Homebrew/brew or Homebrew/core):
/usr/local/Homebrew/Library/Taps/aybabtme/homebrew-tap/humanlog.rb:9

Hello!,

Where could I PR to either document that --skip-unchanged defaults to true and duplicate key/values are removed on a naked run, or change it so that it is not default functionality?

Thanks!

Whenever I run something like apt update or apt upgrade, there's a warning in the dpkg output:

dpkg-query: warning: parsing file '/var/lib/dpkg/status' near line 10716 package 'humanlog':
 missing 'Maintainer' field

And inspecting the generated .deb package:

$ curl -LkO https://github.com/aybabtme/humanlog/releases/download/0.5.0/humanlog_0.5.0_linux_amd64.deb
$ ar x humanlog_0.5.0_linux_amd64.deb
$ cat control
Package: humanlog
Version: 0.5.0
Section: 
Priority: 
Architecture: amd64
Installed-Size: 3076
Description: no description given

It seems that the Maintainer field is indeed not set.

It looks like goreleaser can set this when it generates the package: https://goreleaser.com/customization/nfpm/

As title says, I suggest you guys start using travis-ci with GoReleaser, it lets you cross build binaries to your releases.