Comments (6)
osiegmar
commented on July 21, 2024
1
Thanks for bringing this up. It seems this is a long-standing issue within Graylog itself – see Graylog2/graylog2-server#873.
As the GELF protocol does not contain any restrictions on the length of key-value pairs, I don't want to add trimming functionality to logback-gelf. I consider 32K values as a rare edge case, and I would recommend to handle this in your application.
You could create a custom MDC mapper:
public class CustomMdcDataFieldMapper implements GelfFieldMapper<String> {
private static final int MAX_LENGTH = 20_000;
@Override
public void mapField(ILoggingEvent event, BiConsumer<String, String> valueHandler) {
Map<String, String> mdcPropertyMap = event.getMDCPropertyMap();
if (mdcPropertyMap != null) {
mdcPropertyMap.forEach((k, v) -> valueHandler.accept(k, trimToLength(v)));
}
}
private static String trimToLength(String value) {
return value.length() > MAX_LENGTH ? value.substring(0, MAX_LENGTH) : value;
}
}And then use it in your logback configuration:
<configuration>
<appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
<graylogHost>localhost</graylogHost>
<graylogPort>12201</graylogPort>
<encoder class="de.siegmar.logbackgelf.GelfEncoder">
<includeMdcData>false</includeMdcData>
<fieldMapper class="your.package.CustomMdcDataFieldMapper"/>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="GELF" />
</root>
</configuration>I didn't have a chance to test this, but it should work.
from logback-gelf.
ntngel1
commented on July 21, 2024
Thank you for solution, didn't even know that it's possible to write such custom field mapper that will handle such cases. Definitely going to use it.
from logback-gelf.
ntngel1
commented on July 21, 2024
Well, I tried this and can't really make it work. I have following config:
<?xml version="1.0" encoding="UTF-8"?>
<!-- https://www.playframework.com/documentation/latest/SettingsLogger -->
<configuration>
<conversionRule conversionWord="coloredLevel" converterClass="play.api.libs.logback.ColoredLevel" />
<include resource="shared/console.xml"/>
<appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
<graylogHost>${graylog.server}</graylogHost>
<graylogPort>${graylog.port}</graylogPort>
<encoder class="de.siegmar.logbackgelf.GelfEncoder">
<originHost>${graylog.source}</originHost>
<includeRawMessage>false</includeRawMessage>
<includeMarker>true</includeMarker>
<includeCallerData>false</includeCallerData>
<includeRootCauseData>false</includeRootCauseData>
<includeLevelName>false</includeLevelName>
<includeKeyValues>false</includeKeyValues>
<fieldMapper class="utils.logging.TestMapper"/>
<shortMessageLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>%.-${graylog.maxMessageLength}m%nopex</pattern>
</shortMessageLayout>
<fullMessageLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>%.-${graylog.maxMessageLength}m%n</pattern>
</fullMessageLayout>
<numbersAsString>false</numbersAsString>
</encoder>
</appender>
<logger name="play" level="INFO" />
<logger name="application" level="DEBUG" />
<logger name="reactivemongo" level="INFO" />
<appender name="ASYNCSTDOUT" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="STDOUT" />
</appender>
<root level="WARN">
<appender-ref ref="ASYNCSTDOUT" />
<appender-ref ref="GELF" />
</root>
</configuration>Mapper:
package utils.logging;
import ch.qos.logback.classic.spi.ILoggingEvent;
import de.siegmar.logbackgelf.GelfFieldMapper;
import java.util.function.BiConsumer;
public class TestMapper implements GelfFieldMapper<String> {
@Override
public void mapField(ILoggingEvent event, BiConsumer<String, String> valueHandler) {
}
}Stacktrace:
18:53:52,265 |-ERROR in ch.qos.logback.core.model.processor.ImplicitModelHandler - Could not create component [fieldMapper] of type [utils.logging.TestMapper] java.lang.ClassNotFoundException: utils.logging.TestMapper
at java.lang.ClassNotFoundException: utils.logging.TestMapper
at at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
at at kamon.instrumentation.sbt.KanelaOnSystemClassLoader.loadClass(KanelaOnSystemClassLoader.java:33)
at at kamon.instrumentation.sbt.play.SbtKanelaRunnerPlay$SbtKanelaClassLoader.loadClass(SbtKanelaRunnerPlay.scala:77)
at at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at at ch.qos.logback.core.util.Loader.loadClass(Loader.java:132)
at at ch.qos.logback.core.model.processor.ImplicitModelHandler.doComplex(ImplicitModelHandler.java:134)
at at ch.qos.logback.core.model.processor.ImplicitModelHandler.handle(ImplicitModelHandler.java:94)
at at ch.qos.logback.core.model.processor.DefaultProcessor.secondPhaseTraverse(DefaultProcessor.java:241)
at at ch.qos.logback.core.model.processor.DefaultProcessor.secondPhaseTraverse(DefaultProcessor.java:253)
at at ch.qos.logback.core.model.processor.DefaultProcessor.secondPhaseTraverse(DefaultProcessor.java:253)
at at ch.qos.logback.core.model.processor.DefaultProcessor.secondPhaseTraverse(DefaultProcessor.java:253)
at at ch.qos.logback.core.model.processor.DefaultProcessor.traversalLoop(DefaultProcessor.java:90)
at at ch.qos.logback.core.model.processor.DefaultProcessor.process(DefaultProcessor.java:106)
at at ch.qos.logback.core.joran.GenericXMLConfigurator.processModel(GenericXMLConfigurator.java:216)
at at ch.qos.logback.core.joran.GenericXMLConfigurator.doConfigure(GenericXMLConfigurator.java:178)
at at ch.qos.logback.core.joran.GenericXMLConfigurator.doConfigure(GenericXMLConfigurator.java:123)
at at ch.qos.logback.core.joran.GenericXMLConfigurator.doConfigure(GenericXMLConfigurator.java:66)
at at ch.qos.logback.classic.util.DefaultJoranConfigurator.configureByResource(DefaultJoranConfigurator.java:68)
at at play.api.libs.logback.LogbackLoggerConfigurator.configure(LogbackLoggerConfigurator.scala:128)
at at play.api.libs.logback.LogbackLoggerConfigurator.configure(LogbackLoggerConfigurator.scala:75)
at at di.CashboxOnlineComponents.$anonfun$new$1(CashboxOnlineComponents.scala:22)
at at di.CashboxOnlineComponents.$anonfun$new$1$adapted(CashboxOnlineComponents.scala:22)
at at scala.Option.foreach(Option.scala:437)
at at di.CashboxOnlineComponents.<init>(CashboxOnlineComponents.scala:22)
at at ApplicationLoader.load(ApplicationLoader.scala:15)
at at play.core.server.DevServerStart$DevServerApplicationProvider$1.$anonfun$reload$2(DevServerStart.scala:233)
at at play.utils.Threads$.withContextClassLoader(Threads.scala:22)
at at play.core.server.DevServerStart$DevServerApplicationProvider$1.reload(DevServerStart.scala:225)
at at play.core.server.DevServerStart$DevServerApplicationProvider$1.get(DevServerStart.scala:190)
at at play.core.server.AkkaHttpServer.handleRequest(AkkaHttpServer.scala:320)
at at play.core.server.AkkaHttpServer.$anonfun$createServerBinding$1(AkkaHttpServer.scala:224)
at at akka.stream.impl.fusing.MapAsyncUnordered$$anon$31.onPush(Ops.scala:1430)
at at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:542)
at at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:423)
at at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:650)
at at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:521)
at at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:625)
at at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:800)
at at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:818)
at at akka.actor.Actor.aroundReceive(Actor.scala:537)
at at akka.actor.Actor.aroundReceive$(Actor.scala:535)
at at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:716)
at at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579)
at at akka.actor.ActorCell.invoke(ActorCell.scala:547)
at at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
What am I doing wrong?
from logback-gelf.
osiegmar
commented on July 21, 2024
What am I doing wrong?
Hard to say, stacktrace is incomplete
from logback-gelf.
ntngel1
commented on July 21, 2024
Oops, missed one line, edited my previous message ⏫
from logback-gelf.
osiegmar
commented on July 21, 2024
As the exception says – your class cannot be found on classpath. Based on the given information I cannot see why this happens. You may want to create a very basic example that can be shared via public Github repository...
from logback-gelf.
Related Issues (20)
- All newlines in short messages are replaced with whitespaces HOT 10
- GelfHttpAppender does not set Uri HOT 1
- Markers are sent with Brackets. Like "[MARKER_NAME]" instead of "MARKER_NAME" without brackets HOT 1
- It reports only Sl4j logs not console logs HOT 3
- Macbook GelfUdpAppender garbled HOT 2
- GZIP Compression with GelfUdpAppender HOT 1
- Update logback-classic to 1.2.8 HOT 3
- Logback-classic release 1.2.9
- tcp configuration error HOT 3
- Add ability to log thread ID in addition to thread name HOT 2
- Enable adding multiple static fields via environment variable and/or logback.properties HOT 2
- Mutual TLS fails, HOT 1
- Add another method for adding static field to GelfEncoder HOT 2
- GelfMessage should fail on host = "", shortmessage="" HOT 4
- Add ability to log formatted time HOT 1
- Add use of structured arguments HOT 1
- JSON Encoding Performance HOT 1
- Several minor improvements HOT 1
- Faster MessageIdSupplier with less object allocation, error-prone HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logback-gelf.