phuslu / log Goto Github PK

View Code? Open in Web Editor NEW

635.0 23.0 43.0 803 KB

Fastest structured logging

License: MIT License

Go 99.31% Assembly 0.69%

structured-logging json golang logging log logger slog

log's People

Contributors

Stargazers

Watchers

log's Issues

Disable JSON in log.FileWriter

I have trying to implement logger in my app, have few things to be cleared as those are not present in readme or didn't get it how to do.
Those are:

Disable JSON format and log like nginx log in log.FileWriter or like its there in ConsoleWriter basically formating in FileWriter
Write JSON to console, can this be achieved by ConsoleWriter.Formatter
It has been mentioned for log.MultiWriter that It can write level to different file, can we have custom/default log level and lock file to level only in that file or console.
readme mentioned this log lib is inspired by lumberjack also but can lumberjack be used to rotate log file in log.FileWriter. Not sure about performance with cron for rotating logs.

[QUESTION] - How to use graylog with this library

I'm trying to use Graylog as writer. Instead of logging as individual field log is printed as single string field.

Graylog library: https://github.com/Graylog2/go-gelf

How would I implement it for graylog

The code I'm using is this

gelfWriter, err := gelf.NewUDPWriter(grayLogAddr)
	if err != nil {
		panic(err)
	}
	log.DefaultLogger = log.Logger{
		Level:      log.InfoLevel,
		Caller:     0,
		TimeField:  "date",
		TimeFormat: "2006-01-02 15:04:05",
		Writer:     &log.IOWriter{gelfWriter},
	}
r := Request{
			requestId: rid,
			method:    c.Method(),
			host:      c.Hostname(),
			path:      c.Path(),
			request:   string(payload),
			response:  string(response),
			status:    c.Response().StatusCode(),
			latency:   fmt.Sprintf("%s", time.Since(start)),
		}
		log.Info().EmbedObject(&r).Msg("")

log.Fatal() 会崩溃

请运行这个：
package main import "github.com/phuslu/log" func main() { log.Fatal().Str("foo", "bar").Int("number", 42).Msg("a structured logger") }
or https://play.golang.org/p/mapv1Ferz8T
tks

io_uring support for FileWriter

Currently, go runtime still has not support io_uring, maybe we could have a try on

https://github.com/hodgesds/iouring-go#benchmarks

RingbufferWriter is needed

Currently, ringbuffer implementation in golang is not good enough, we should introduce a ringbuffer writer from scratch.

Gzip examples writes truncated file

Following the example in #19

async := &log.AsyncWriter{
    ChannelSize: 1,
    Writer: log.IOWriter{
        Writer: gzip.NewWriter(&log.FileWriter{
            Filename:   "foo.log.gz",
            MaxSize:    50 * 1024 * 1024,
            MaxBackups: 7,
        }),
    },
}
logger := log.Logger{
    Writer: async,
}
logger.Info().Msg("hello phuslu")
async.Close()

ls -la
total 40
-rw-r--r--  1 cgrindst  staff    10B May 20 11:07 foo.log.2022-05-20T15-07-25.gz
lrwxr-xr-x  1 cgrindst  staff    30B May 20 11:07 foo.log.gz@ -> foo.log.2022-05-20T15-07-25.gz

file foo.log.2022-05-20T15-07-25.gz 
foo.log.2022-05-20T15-07-25.gz: gzip compressed data, truncated

gunzip foo.log.2022-05-20T15-07-25.gz 
gunzip: foo.log.2022-05-20T15-07-25.gz: unexpected end of file
gunzip: foo.log.2022-05-20T15-07-25.gz: uncompress failed

[FEATURE REQUEST] - Add support for function With to predefine parameters for logger

Logging same multiple parameters in various places, we have to repeat the parameters in multiple places. It would be really helpful if the package supports some function that allows to predefine the parameters and use the logger.

For example

logger := logger.With().Str("attr1", "1").Str("clientID","123")
logger.Info().Msg("This is test")

This would print log with provided parameters

{"attr1": "1", "clientID": "123", "msg": "This is test"}

MultiWriter panic with AsyncWriter

Description: Got panic when trying to print messages on Logger with MultiWriter: AsyncWriter in ErrorWriter, InfoWriter, WarnWriter struct fields, and ConsoleWriter.
Reproduction example: https://play.golang.org/p/gkNrfO2M0Jv

Is it possible to instrument with OpenTelemetry?

How about supporting global level?

Setting level to a specific logger is not very useful, normally we want to change level for all created logger. There is no way to do it currently.

enable gzip support to FileWriter

introduce extra CPU latency maybe save IO latency, give a try？

gin logger documentation: what is ginlogger.Config?

The documentation for the gin logger mentions this:

	// Custom logger
	r.Use(ginlogger.SetLogger(ginlogger.Config{
		Logger: &log.Logger{
			Writer: &log.FileWriter{
				Filename: "access.log",
				MaxSize:  1024 * 1024 * 1024,
			},
		},
		Context: log.NewContext(nil).Str("foo", "bar").Value(),
		Skip:    func(c *gin.Context) bool {
			if c.Request.URL.Path == "/backdoor" {
				return true
			}
			return false
		},
	}))

What is ginlogger.Config ?

The documentation mentioned is at:

https://github.com/phuslu/log-contrib/tree/master/gin

Callers - vary stack depth

Hi @phuslu

I have a case where I'm wrapping logger inside another abstraction, and I want caller to log the point at which the abstraction is called, rather than Info() etc.

How about making the Caller member actually by the number of stack frames to look back instead of just a switch?
e.g. 2/-2 would look one frame further. Is there a better way than simply overloading the meaning of Caller? For instance a new property in case existing users are relying on Caller simply being not zero and not explicitly 1/-1

I could look into it

Disable json for file log

Is it possible writes plain text to file like pretty console log?

v1.0.75 tag 名称是否需要调整一下?

invalid version: unknown revision v1.0.75

AsyncWriter should supports writev

Two step,

add writev support in FileWriter on linux.
add writev in AsyncWriter base on step 1

A draft already pushed to branch writev https://github.com/phuslu/log/blob/writev/async_writev.go

W3C extended log file format fromatter

With very little effort I was able to implement a custom Formatter to make the ConsoleWriter write log lines in a W3C Extended Log File Format compliant way (see: https://www.w3.org/TR/WD-logfile.html)

Although JSON structured logging is all the rage today (and, in fact, I personally prefer JSON big time) there's a lot of log analyzers out there that are unable to read JSON, but are perfectly capable of importing W3C log files.

The problem is that there is no Formatter in the FileWriter writer, which is exactly where it would need to be, if we are to be able to save log files in W3C format.

The way I see it, this could be achieved 2 ways:

add a Formatter to the existing FileWriter interface
create a whole new W3CLogFileWriter interface (probably not a good idea)

Also, W3C log files require a "header" to be written at the beginning of every file, so ideally each time the file is rotated this header should be automatically written to the file, before we can actually add log lines to it.

Here's a W3C log file header example (taken from Microsoft IIS):

#Software: Internet Information Services 8.0 
#Version: 1.0 
#Date: 2021-05-02 17:42:15 
#Fields: time c-ip cs-method cs-uri-stem sc-status cs-version

This could probably be achieved with some custom wizardry in the Cleaner function of the FileWriter interface, but while we're at it, why not make everyone life's easier, and provide a string property for it, and have the Writer automagically add it at the beginning of each new file, right when the log its rotated?

JSON logging is still far far better, I think we can all agree on that. But W3C log file format is not dead, and is required by more users than I originally thought possible. Any feedback on this would be greatly appreciated. Thank you in advance.

Panic on using for aws s3

project code here

http://github.com/pankajsoni19/go-kafka-to-s3


admin@ip-41-2-0-28:~/go-kafka-to-s3$ panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x7891ed]

goroutine 140 [running]:
github.com/phuslu/log.(*FileWriter).Write(0x0, 0xc000f78000, 0x99, 0xa0, 0x98, 0xc000f78000, 0x18)
	/home/admin/go/pkg/mod/github.com/phuslu/[email protected]/file.go:92 +0x2d
log.(*Logger).Output(0xc000082cd0, 0x2, 0xc0000ae300, 0x80, 0x0, 0x0)
	/usr/local/go/src/log/log.go:181 +0x284
log.(*Logger).Printf(0xc000082cd0, 0xc0000ac1b0, 0x2e, 0xc000150320, 0x1, 0x1)
	/usr/local/go/src/log/log.go:188 +0x7e
github.com/aws/smithy-go/logging.StandardLogger.Logf(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/logging/logger.go:74
github.com/aws/aws-sdk-go-v2/aws/retry.Attempt.logf(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/retry/middleware.go:59
github.com/aws/aws-sdk-go-v2/aws/retry.Attempt.handleAttempt(0xc7b101, 0xc7bde0, 0xc0000cc420, 0xc04c90, 0xc7b1e0, 0xc000138b40, 0xbd4620, 0xc000138a50, 0xc728a0, 0xc00000e560, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/retry/middleware.go:150 +0x9af
github.com/aws/aws-sdk-go-v2/aws/retry.Attempt.HandleFinalize(0xed7b58701, 0xc7bde0, 0xc0000cc420, 0xc04c90, 0xc7b1e0, 0xc000138a20, 0xbd4620, 0xc000138960, 0xc728a0, 0xc00000e560, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/retry/middleware.go:86 +0x305
github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_finalize.go:200
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding.(*DisableGzip).HandleFinalize(0x121ae80, 0xc7b1e0, 0xc000138a20, 0xbd4620, 0xc000138960, 0xc728a0, 0xc00000e580, 0xc00000e5a0, 0xc000f577d0, 0xc00000e5a0, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/internal/[email protected]/accept_encoding_gzip.go:67 +0x167
github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_finalize.go:200
github.com/aws/smithy-go/middleware.(*FinalizeStep).HandleMiddleware(0xc0000b2048, 0xc7b1e0, 0xc000138a20, 0xbd4620, 0xc000138960, 0xc728c0, 0xc00000e2a0, 0xa81a664ba2bfe8a1, 0xc76c51a3c24b8b70, 0xb8cb68db0a658c3c, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_finalize.go:114 +0x1b7
github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/middleware.go:57
github.com/aws/smithy-go/middleware.buildWrapHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:184
github.com/aws/aws-sdk-go-v2/aws/middleware.(*requestUserAgent).HandleBuild(0xc0001500d0, 0xc7b1e0, 0xc000138a20, 0xbd4620, 0xc000138960, 0xc72840, 0xc0001501d0, 0x4461900b0dca02ef, 0x10, 0xb51260, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/middleware/user_agent.go:232 +0x3b7
github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:200
github.com/aws/aws-sdk-go-v2/aws/signer/v4.(*contentSHA256Header).HandleBuild(0x121ae80, 0xc7b1e0, 0xc000138a20, 0xbd4620, 0xc000138960, 0xc72860, 0xc00000e4a0, 0x121ae80, 0xb6e4c0, 0xc0001389f0, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/signer/v4/middleware.go:195 +0x1dd
github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:200
github.com/aws/aws-sdk-go-v2/aws/signer/v4.(*computePayloadSHA256).HandleBuild(0x121ae80, 0xc7b1e0, 0xc000138870, 0xbd4620, 0xc000138960, 0xc72860, 0xc00000e4c0, 0x0, 0x0, 0x7f65e4342500, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/signer/v4/middleware.go:156 +0x27c
github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:200
github.com/aws/smithy-go/transport/http.(*ComputeContentLength).HandleBuild(0x121ae80, 0xc7b1e0, 0xc000138870, 0xbd4620, 0xc000138960, 0xc72860, 0xc00000e4e0, 0x237b1360e7940a64, 0xc000f5a160, 0xe91575e0efd69d15, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/transport/http/middleware_content_length.go:55 +0x17f
github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:200
github.com/aws/aws-sdk-go-v2/aws/middleware.ClientRequestID.HandleBuild(0xc7b1e0, 0xc000138870, 0xbd4620, 0xc000138960, 0xc72860, 0xc00000e500, 0x20, 0x20, 0x7f66176217d0, 0xc00000e520, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/middleware/middleware.go:42 +0x1ed
github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:200
github.com/aws/smithy-go/middleware.(*BuildStep).HandleMiddleware(0xc0000b2040, 0xc7b1e0, 0xc000138870, 0xbd4620, 0xc000138960, 0xc728c0, 0xc00000e2c0, 0xc000f580a0, 0x0, 0x0, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_build.go:114 +0x1b7
github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/middleware.go:57
github.com/aws/smithy-go/middleware.serializeWrapHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:192
github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*removeBucketFromPathMiddleware).HandleSerialize(0x121ae80, 0xc7b1e0, 0xc000138870, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138960, 0xc72980, 0xc000150140, 0xc0001161b0, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/internal/customizations/remove_bucket_middleware.go:27 +0x27a
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*updateEndpoint).HandleSerialize(0xc00000e220, 0xc7b1e0, 0xc000138870, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138960, 0xc72900, 0xc00000e3c0, 0xc000138840, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/internal/customizations/update_endpoint.go:175 +0x213
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/aws-sdk-go-v2/service/internal/s3shared.(*EnableDualstack).HandleSerialize(0xc00000e1e0, 0xc7b1e0, 0xc000138870, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138960, 0xc72900, 0xc00000e3e0, 0x1, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/internal/[email protected]/update_endpoint.go:75 +0x1b1
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/aws-sdk-go-v2/service/s3.(*awsRestxml_serializeOpPutObject).HandleSerialize(0x121ae80, 0xc7b1e0, 0xc000138870, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138720, 0xc72900, 0xc00000e400, 0xc000138840, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/serializers.go:6936 +0x3a9
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*processARNResource).HandleSerialize(0xc00000e1c0, 0xc7b1e0, 0xc000138870, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138720, 0xc72900, 0xc00000e420, 0x121ae80, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/internal/customizations/process_arn_resource.go:48 +0x1382
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/aws-sdk-go-v2/service/s3.(*ResolveEndpoint).HandleSerialize(0xc00000e140, 0xc7b1e0, 0xc0001386f0, 0xb3d4e0, 0xc0000aa140, 0xbd4620, 0xc000138720, 0xc72900, 0xc00000e440, 0x7a0b25, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/endpoints.go:111 +0x5c7
github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:208
github.com/aws/smithy-go/middleware.(*SerializeStep).HandleMiddleware(0xc000150080, 0xc7b1e0, 0xc0001386f0, 0xb3d4e0, 0xc0000aa140, 0xc728c0, 0xc00000e2e0, 0x3e8000081a4, 0x3e8, 0x203000, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_serialize.go:122 +0x1fb
github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/middleware.go:57
github.com/aws/smithy-go/middleware.initializeWrapHandler.HandleInitialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:184
github.com/aws/aws-sdk-go-v2/service/s3.(*validateOpPutObject).HandleInitialize(0x121ae80, 0xc7b1e0, 0xc0001386f0, 0xb3d4e0, 0xc0000aa140, 0xc72960, 0xc0001500f0, 0x549046, 0xbafa40, 0xc0001386f0, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/validators.go:1630 +0xdb
github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:200
github.com/aws/smithy-go/middleware.(*setLogger).HandleInitialize(0xc0001500c0, 0xc7b1e0, 0xc0001386c0, 0xb3d4e0, 0xc0000aa140, 0xc728e0, 0xc00000e340, 0x121ae80, 0xb6e4c0, 0xc000138690, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/logging.go:45 +0xb5
github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:200
github.com/aws/aws-sdk-go-v2/aws/middleware.RegisterServiceMetadata.HandleInitialize(0xbe1fde, 0x2, 0xbe20a8, 0x2, 0xc0000b1ee6, 0x9, 0xbe4466, 0x9, 0xc7b1e0, 0xc0001382a0, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/aws/middleware/metadata.go:40 +0xe7
github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:200
github.com/aws/aws-sdk-go-v2/service/internal/s3shared.(*ARNLookup).HandleInitialize(0xc0000b2058, 0xc7b1e0, 0xc0001382a0, 0xb3d4e0, 0xc0000aa140, 0xc728e0, 0xc00000e380, 0xc00000e3a0, 0xc000f497c0, 0xc00000e3a0, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/internal/[email protected]/arn_lookup.go:39 +0xca
github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:200
github.com/aws/smithy-go/middleware.(*InitializeStep).HandleMiddleware(0xc0000b2038, 0xc7b1e0, 0xc0001382a0, 0xb3d4e0, 0xc0000aa140, 0xc728c0, 0xc00000e300, 0x0, 0xc728c0, 0xc00000e300, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/step_initialize.go:114 +0x1b7
github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/middleware.go:57
github.com/aws/smithy-go/middleware.(*Stack).HandleMiddleware(0xc0000c8000, 0xc7b1e0, 0xc0001382a0, 0xb3d4e0, 0xc0000aa140, 0xc729a0, 0xc0001500e0, 0x0, 0xc729a0, 0xc0001500e0, ...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/stack.go:109 +0x151
github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...)
	/home/admin/go/pkg/mod/github.com/aws/[email protected]/middleware/middleware.go:57
github.com/aws/aws-sdk-go-v2/service/s3.(*Client).invokeOperation(0xc000130dc0, 0xc7b160, 0xc0000b0018, 0xbe4466, 0x9, 0xb3d4e0, 0xc0000aa140, 0x0, 0x0, 0x0, ...)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/api_client.go:165 +0x533
github.com/aws/aws-sdk-go-v2/service/s3.(*Client).PutObject(0xc000130dc0, 0xc7b160, 0xc0000b0018, 0xc0000aa140, 0x0, 0x0, 0x0, 0x12, 0xc00045fea0, 0xc00045fed8)
	/home/admin/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/[email protected]/api_op_PutObject.go:82 +0xfb
main.upload(0xc000143860, 0xc0000b1eb0, 0xb, 0xc0000ac0f0, 0x27, 0x0, 0x0)
	/home/admin/go-kafka-to-s3/main.go:235 +0x34c
main.setupRotation.func2.1(0xc0000ac060, 0x24, 0xc000143860, 0xc0000b1eb0, 0xb)
	/home/admin/go-kafka-to-s3/main.go:192 +0x172
created by main.setupRotation.func2
	/home/admin/go-kafka-to-s3/main.go:182 +0x1cd
^C
[1]+  Exit 2                  ./go-kafka-to-s3

[Question] How can I implement this library with other libraries like Gorm v2?

Currently Gorm v2 is using it's default logger.
I want to use this library for Gorm v2.

How can I implement this library?

Here's the doc they are suggesting to implement but can't figure out how I could implement this log

Time() always format with local timezone

I notice when using func (e *Entry) Time(key string, t time.Time) *Entry , t always formatted using my local time zone, even if t has location set to UTC.
For comparison, zerolog will format with UTC if t has location set to UTC

log With()

Great logger, simple and easy to use. Are you looking to add a with() function, as this is most important when logging web transactions and correlation of single web requests, through the core application.

Thanks,
Bob

清理旧备份时,匹配备份文件名称没有关联TimeFormat配置.可造成旧备份未被清理

# file.go:175
		ext := filepath.Ext(filename)
		pattern := filename[0:len(filename)-len(ext)] + ".20*" + ext
		if names, _ := filepath.Glob(pattern); len(names) > 0 {
			sort.Strings(names)
			for i := 0; i < len(names)-backups-1; i++ {
				os.Remove(names[i])
			}
		}
	}(oldfile, w.file.Name(), w.Filename, w.MaxBackups, w.ProcessID)

启动多个应用程序写入同一日志文件

&log.FileWriter{
    Filename:     "logs/access.log",
    FileMode:     0600,
    MaxSize:      100 << 20,
    MaxBackups:   7,
    EnsureFolder: true,
    LocalTime:    true,
    TimeFormat:   "run",
},

lrwxrwxrwx 1 root root   14 Dec 28 13:50 access.log -> access.run.log*
-rwxrwxrwx 1 root root 1549 Dec 28 13:52 access.run.log*

配置如上. 主要为了实现像 nginx 一样, 把所有日志都写在相同文件名中.
请问我开启了多个应用程序时, 会有多个进程同时写相同的日志文件, 有问题吗?
或者有什么进程安全参数设置项吗?
谢谢!

Add golangci-lint

There is some issues in running golangci-lint on this project. I am getting the same error output on my local machine as the job logs.

I have created an issue in golangci/golangci-lint. For now, let's wait until the issue gets resolved, then we add golangci-lint again.

Please help to confirm whether rfc5424 is supported？

I'm a new user of phuslu/log, I want to know if SyslogWriter supports RFC5424? Thanks.
https://www.rfc-editor.org/rfc/rfc5424

[FEATURE REQUEST] - Allow logging in CSV format

Is it possible to log in CSV format? if not maybe add the feature to add CSV

is cbor logger needed?

Currently, there're Logger(for json) and TSVLogger(for tsv/csv) in this lib.
I'm not sure a cbor logger is needed or not, if you have strong reasons, please comment.

it does not rotate if you have a TimeFormat defines and a MaxSize then reach MaxSize with the same TimeFormat

I would like to comment or expand FileWriter so we can al better understand it.

Files are saved according to name.timestamp.ext.
File is always created with the defined or default TimeFormat.
Once program is initiated it only autorotates when condition for MaxSize is meet and has a different name.
If the filename is the same it never autorotates (Should this be like this?)
If TimeFormat is default and goes by minutes each minute it generates a new log file. (Should it be like this?)
There is no way to compres. (Maybe we can implement it or add AfterRotate func for user to do what he wants)

I thinks this deserves a thought to see if it marches what people usually expect.

When rotating if file exists and MaxSize is meet we should append _n
Don't know how to go about file name... if it does not meet MaxSize should we use the same name whether it takes 30m or 30 days and only add timestamp on rotate?

w.vwriter undefined (type *AsyncWriter has no field or method vwriter

/../../../../go/pkg/mod/github.com/phuslu/[email protected]/async.go:52:9: w.vwriter undefined (type *AsyncWriter has no field or method vwriter) (exit status 1)

Getting this go version 1.22

stdout/stderr based on log level

Hi @phuslu

Here's another suggestion 😁

Some log forwarders like filebeat can switch on the stream on which logs are emitted. It would be nice if for the standard logger type we could select the output stream based on the log level, e.g. severity of warning or higher goes to stderr and lower levels go to stdout. User can set at which level the change from stdout to stderr happens.

Disabling compiler optimization and linting breaks phuslu

Hello,

with the latest release (102) when I build my project with

go build -gcflags="all=-N -l" -o ./test

disabling compiler optimization and linting, the logger breaks at compile time with:

Build Error: ...
github.com/phuslu/log.(*Logger).header: nosplit stack over 792 byte limit
github.com/phuslu/log.(*Logger).header<1>
    grows 1848 bytes, calls sync.(*Pool).Get<1>
        grows 8 bytes, calls runtime.morestack<0>
        1064 bytes over limit
    grows 1848 bytes, calls runtime.gcWriteBarrier2<1>
        grows 8 bytes, calls gcWriteBarrier<22>
            grows 128 bytes, calls runtime.wbBufFlush<0>
                grows 8 bytes, calls runtime.wbBufFlush<1>
                    grows 24 bytes, calls runtime.puintptr.ptr<1>
                    1224 bytes over limit
                    grows 24 bytes, calls runtime.(*wbBuf).discard<1>
                    1224 bytes over limit
                    grows 24 bytes, calls runtime.systemstack<0>
                        grows 16 bytes, calls gosave_systemstack_switch<22>
                            grows 8 bytes, calls runtime.abort<0>
                            1248 bytes over limit
                        grows 16 bytes, calls indirect
                            grows 8 bytes, calls runtime.morestack<0>
                            1248 bytes over limit
                        grows 8 bytes, calls indirect
                            grows 8 bytes, calls runtime.morestack<0>
                            1240 bytes over limit
    grows 1848 bytes, calls runtime.convT<1>
        grows 8 bytes, calls runtime.morestack<0>
        1064 bytes over limit
    grows 1848 bytes, calls runtime.gcWriteBarrier2<1>
        grows 8 bytes, calls gcWriteBarrier<22>
            grows 128 bytes, calls runtime.wbBufFlush<0>
                grows 8 bytes, calls runtime.wbBufFlush<1>
                    grows 24 bytes, calls runtime.puintptr.ptr<1>
                    1224 bytes over limit
                    grows 24 bytes, calls runtime.(*wbBuf).discard<1>
                    1224 bytes over limit
                    grows 24 bytes, calls runtime.systemstack<0>
                        grows 16 bytes, calls gosave_systemstack_switch<22>
                            grows 8 bytes, calls runtime.abort<0>
[...]

I suspect it's due to commit: 8332496

Note this breaks debugging with vscode and delve. Previous version of phuslu (101) works fine.

Incorrect caller when using slog

Hi!

package main

import (
	"log/slog"

	plog "github.com/phuslu/log"
)

func main() {
	plog.DefaultLogger = plog.Logger{
		Level: plog.DebugLevel,
		Caller: 1,
	}

	var logger *slog.Logger
	logger = plog.DefaultLogger.Slog()

	plog.Info().Msg("msg from phuslu/log")
	logger.Info("info msg from slog")
	logger.Error("error msg from slot")
}

Output:

{"time":"2023-09-06T15:36:41.406+08:00","level":"info","caller":"main.go:18","goid":1,"message":"msg from phuslu/log"}
{"time":"2023-09-06T15:36:41.406+08:00","caller":"logger_go1.21.go:30","goid":1,"message":"info msg from slog"}
{"time":"2023-09-06T15:36:41.406+08:00","caller":"logger_go1.21.go:30","goid":1,"message":"error msg from slot"}

And missing level when using log/slog, it would be great to have support for slog levels!

FileWriter should be lockfree

*os.File.Write already has a lock for concurrency call. so we could remove current mutex of FileWriter.
A original but outdated implementation reverted at b6dc7fe

FileWriter comment mentions MaxAge but that field does not exist

The comment for FileWriter mentions MaxAge but that field does not exist.

Any files with an encoded timestamp older than MaxAge days are deleted, regardless of MaxBackups

MaxAge would be nice to have if this library is meant to be a drop-in replacement for Lumberjack.

[FEATURE REQUEST] - Hide and/or remove PII data from logging

Description

struct or map might contain PII data and look for an option to hide(mask) such PII data and/or remove such keys

有内置日志采样功能的计划吗?

// Will let 5 debug messages per period of 1 second.
// Over 5 debug message, 1 every 100 debug messages are logged.
// Other levels are not sampled.
sampled := log.Sample(zerolog.LevelSampler{
    DebugSampler: &zerolog.BurstSampler{
        Burst: 5,
        Period: 1*time.Second,
        NextSampler: &zerolog.BasicSampler{N: 100},
    },
})
sampled.Debug().Msg("hello world")

// Output: {"time":1494567715,"level":"debug","message":"hello world"}

您好吖.

很喜欢干净整洁的库, 高性能无依赖是天花板. 功能也很全面.

不过, 看了一圈没发现上面的日志采样功能, 在一些高并发场景我们可能更希望1秒有几个日志即可. 或意外产生错误时很可能疯狂记录错误日志, 此时也希望仅采样错误来记录.

如果已有此功能或简单方案, 请贴下范例.

若无, 请问有计划吗?

我感觉这样就完全可以替代我心中理想的日志库了.

谢谢~~

How to use it with echo ?

I am on echo v4

Logger = log.Logger{
		Level: log.InfoLevel,
		Writer: &log.AsyncWriter{
			ChannelSize: 100,
			Writer:      fileWriter,
		},
	}

logConfig := middleware.DefaultLoggerConfig
logConfig.Output = Logger   // error here
e.Use(middleware.LoggerWithConfig(logConfig))

Error is

cannot use Logger.Writer (variable of type log.Writer) as io.Writer value in assignment: missing method Writecompiler

MultiWriter based on io.Writer

Sometimes I need to write to different outputs with a selected level.
For example, given a Debug level I would like to be able to write to console, log file and syslog.

Have tried io.MultiWriter but ConsoleWriter does not implement io.Writer method.

Alternatively we could implement something like https://github.com/rs/zerolog/blob/117cb53bc66413d9a810ebed32383e53416347e3/writer.go#L88

I have tried several things and the only option I see is MultiWriter but it's level based not io.Writer based.

Any suggestions?

Docs request: clarification of optimization techniques

I would be interested in a blurb in the documentation somewhere on the techniques you've chosen to go with which provide this library with its performance over the others which you benchmark against. Often there is a tradeoff in terms of functionality or complexity when enhancing speed over alternatives. Alternatively, if there are no such concerns, and it's simply clever engineering, that's worth mentioning too, since it would help users contextualize their choice of this library over the alternatives.

FileWriter Filename

This could be considered as an improvement? maybe?

When you output to file, you have name.timestamp.ext that's ok... but when there are many cases when this is a little bit unconfutable.
I think I should be name.ext and when rotated name.timestamp.ext

If you are testing
If you have a console command options
If you need to stop and start the project a couple of times
...

When rotation happens, because of whatever condition, having timestamp is ok, but if there are no conditions that have been meet it should be just the name and append data.

For compatibility issues we could implement an AutoRotateOnInitialize (bool), or OnlyTimestampOnRotate (bool) a simpler name...

ConsoleWriter cant print Err

use consolewriter will lost Err() info.
like:

package main

import (
	"errors"

	"github.com/phuslu/log"
)

var glog = (&log.Logger{
	Level:      log.InfoLevel,
	Caller:     1,
	TimeFormat: "0102 15:04:05.999999",
	Writer: &log.ConsoleWriter{
		ColorOutput:    false,
		QuoteString:    true,
		EndWithMessage: true,
	},
})

func main() {
	glog.Error().Err(errors.New("an error")).Msgf("hello glog")
}

#output: 
1110 23:00:00 ERR 1 prog.go:21 > hello glog

Phuslu Wrapper for Fiber

Hi, I've created phuslu log wrapper for Fiber. Please suggest if you have any opinions or suggestion to improve

https://github.com/sujit-baniya/sblogger

Msg() truncate strings with single quotes when having a custom Formatter

given a simple Formatter like this :

Formatter: func (w io.Writer, a *log.FormatterArgs) (int, error) {
			return fmt.Fprintf(w, "%s", a.Message)

this :

log.Info().Msg("aaaa 'b' cccc")

will output :

aaaa '

对于FileWriter类型的输出，如何支持自定义格式？

作者你好，貌似对于ConsoleWriter类型才有Formatter方法，若我想在输出文件时也自定义格式，要怎么弄？谢谢

Long filename in `caller`

It would be much convenient if we could log long filename and line number, and stdlib supports this.

Plans for supporting Unstructured Logging

May I know if there are plans to support unstructured logging with async support?

Single Quotation Mark

Given the following example:

var logger = struct {
	Console plog.Logger
	Pretty  plog.Logger
	File    plog.Logger
}{
	Console: plog.Logger{
		Level:  plog.TraceLevel,
		Caller: 1,
		Writer: &plog.IOWriter{os.Stdout},
	},
	Pretty: plog.Logger{
		Level:  plog.TraceLevel,
		Caller: 1,
		Writer: &plog.ConsoleWriter{
			ColorOutput:    true,
			QuoteString:    true,
			EndWithMessage: true,
		},
	},
	File: plog.Logger{
		Level:  plog.TraceLevel,
		Caller: 1,
		Writer: &plog.FileWriter{
			Filename:     "plog_test.log",
			FileMode:     0600,
			MaxSize:      100 * 1024 * 1024,
			MaxBackups:   7,
			EnsureFolder: true,
			LocalTime:    true,
		},
	},
}

func testSpecial() {
	logger.Console.Info().Msg("can't handle single quotation mark")
	logger.Pretty.Info().Msg("can't handle single quotation mark")
	logger.File.Info().Msg("can't handle single quotation mark")
}

I have the following outputs:
Console:
{"time":"2021-06-12T18:10:42.379+02:00","level":"info","caller":"main.go:233","goid":1,"message":"can\u0027t handle single quotation mark"}
Pretty:
2021-06-12T18:10:42.379+02:00 INF 1 main.go:234 > can't handle single quotation mark
File:
{"time":"2021-06-12T18:10:42.379+02:00","level":"info","caller":"main.go:235","goid":1,"message":"can\u0027t handle single quotation mark"}

It seems it can\u0027t handle it correctly... lol

ConsoleWriter needs a faster JSON parser

Per discussion at #8 (comment) and #5 (comment), we need bring a faster JSON parser to mitigate performance concern

如何把所有日志放一个文件

`log.DefaultLogger.Writer = &log.MultiEntryWriter{

	&log.FileWriter{
		Filename:  "main.log",
		MaxSize:   100 << 20,
		LocalTime: true,
	},
}`

每次启动会生一个新的"main.2021-10-11T10-32-41.log" 类似的文件，如何控制，都写在同一个文件下，而不是新生成。

Compatibility with rs/logbench

I like this logger a lot because it has zero dependencies and an intuitive API. Therefore, I hope that it can gain more popularity among the Go community.

I have created a draft pull request rs/logbench#4 in the rs/logbench repository to include this logger. However, this logger currently does not pass the validation of logbench, more details can be found in rs/logbench#4.

To pass the logbench validation, this logger must

Allow timestamp to be disabled
Support different encoder for Time and Duration

phuslu / log Goto Github PK

log's People

Contributors

Stargazers

Watchers

Forkers

log's Issues

Description

Recommend Projects

Recommend Topics

Recommend Org