MDC + Context Propagation Feature Request about slf4j HOT 9 OPEN

@mosesn Thank you for your detailed comments. I have added a link to this issue from Changes under consideration for SLF4J version 2.1.0

I was thinking of something similar to your proposition 2 as an extension mechanism for MDC.

Question: How does proposition 2 help in propagating the MDC context to child threads?
Answer: The MDC extension has its own means to extract its own data as appropriate to the current thread, e.g. extreme lengths mentioned earlier.

Thus, if I understand correctly, the MDC extension mechanism allows the user to plugin an external MDC (mapped diagnostic context) into SLF4J's MDC. This achieves the goal of context propagation indirectly by delegating the problem to the extension but without providing a propagation mechanism in SLF4J.

from slf4j.

@mosesn As for the scary message, developers who wish to override mdcAdapter that they might also need to call LoggerContext.setMDCAdapter. Up until 1.5.1, LoggerContext.setMDCAdapter did not allow overriding the mdcAdapter. Commit 097da9dd54c8 changes this.

from slf4j.

Question: How does proposition 2 help in propagating the MDC context to child threads?
Answer: The MDC extension has its own means to extract its own data as appropriate to the current thread, e.g. extreme lengths mentioned earlier.

Yes, that's exactly right. I'm picturing something like MDC.extend(MDCExtension) that would let a customer register an "external MDC".

@mosesn As for the scary message, developers who wish to override mdcAdapter that they might also need to call LoggerContext.setMDCAdapter. Up until 1.5.1, LoggerContext.setMDCAdapter did not allow overriding the mdcAdapter. Commit 097da9dd54c8 changes this.

Thank you!! I'll definitely use this once it's released.

from slf4j.

@mosesn Would like to try to implement?

from slf4j.

Yep, sure! Is there a branch I should do the work in? And is it acceptable to add a method to Slf4jServiceProvider or MdcAdapter, or would you want me to figure out a way to avoid doing that? (I'm thinking about how to communicate to logback that LoggingContext#mdcAdapter should take the MDCExtension into account)

from slf4j.

I would suggest "branch_2.1.x".

It is OK to add methods to the MDCAdapter interface as long as they have default implementations do something reasonable or in the worst case don't do anything.

from slf4j.

Been looking at this exact question recently. We wrote custom encoders for Logback and Log4j2 to provide standard logging contexts for all applications in our division (> 100 apps). It's been working well except for those few developers using reactive APIs.

My thought for this feature is to update the Log4j API to provide logging overloads that include the reactive context. An equivalent to the MDC pattern, e.g. %Context{foo}, could be used to include context variables in the logging output. I think it would be up to developers to populate that context and pass it into Slf4j calls. This relieves Slf4j from having to invent a very difficult wheel.

One problem, is think, is that the RxJava context is not a standard item and is not uniformly supported by all applications. It seems the standard projects, e.g. RxJava, cannot see a good way forward, with a couple attempts at solving the problem abandoned over some years.

- 2015: https://github.com/ReactiveX/RxJava/issues/2885
- 2019: https://github.com/ReactiveX/RxJava/issues/6484

There is this specific Open Telemetry extension for Logback, which is doing the necessary context copying:
https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/logback/logback-mdc-1.0/library/src/main/java/io/opentelemetry/instrumentation/logback/mdc/v1_0/OpenTelemetryAppender.java#L88-L102

I haven't used RxJava/Reactor much. But I have done a mess of asynchronous/non-blocking code in C/C++ over the years. I don't see much hope for standardizing "context local" propagation. Thus, the best way to go for the foreseeable future is to make the logging API explicit wrt reactive context.

from slf4j.

There is support for context also in reactive/non-blocking approach of processing. At least, project-reactor does have it: https://projectreactor.io/docs/core/release/reference/#context

from slf4j.

There is support for context also in reactive/non-blocking approach of processing. At least, project-reactor does have it: https://projectreactor.io/docs/core/release/reference/#context

Exactly. What is missing is a logging API that will include attributes from that context similarly to how existing APIs use MDC. A reactive diagnostic context, if you will.

I'm thinking the context is the 1st parameter to a set of overloads, e.g.

logger.debug(ctx, "foo - bar: {}, biff: {}", bar, biff);

from slf4j.