jlib-framework / jlib-awslambda-logback Goto Github PK
View Code? Open in Web Editor NEWjlib AWS Lambda SLF4J/Logback Appender
License: Apache License 2.0
jlib AWS Lambda SLF4J/Logback Appender
License: Apache License 2.0
I'm fairly new with logging with Java, and is using the out of the box configuration:
<configuration>
<appender name="awslambda" class="org.jlib.cloud.aws.lambda.logback.AwsLambdaAppender">
<encoder type="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] <%-36X{AWSRequestId:-request-id-not-set-by-lambda-runtime}> %-5level %logger{10} - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="awslambda" />
</root>
</configuration>
I want to change the log level to DEBUG
, and tried changing root level to DEBUG
:
<root level="DEBUG">
<appender-ref ref="awslambda" />
</root>
It enabled the debug logs for AWS SDK's and not for my loggers:
import org.slf4j.LoggerFactory
class Foo() {
private val logger = LoggerFactory.getLogger(Foo::class.java)
}
How can I resolve this? Also, how can I prevent having a DEBUG
log level to AWS SDKs?
Running the OWASP dependency check on version 1.0.0 gives a vulnerability warning
jlib-awslambda-logback-1.0.0.jar (pkg:maven/org.jlib/[email protected], cpe:2.3:a:logback:logback:1.0.0:*:*:*:*:*:*:*) : CVE-2017-5929
which corresponds to this issue CVE-2017-5929
Just using those two dependencies, I am not able to import Logback classes directly
import ch.qos.logback.classic.Level; import ch.qos.logback.classic.Logger;
I want to be able to set the root logger's log level from code in order to change it dynamically by reading the lambda environment variable. This allows me to change the level by just saving the lambda instead of deploying it.
just as a quick FYI, i followed the instructions in this stackoverflow: https://stackoverflow.com/questions/46080448/logging-in-aws-lambda-with-slf4j
i kept getting this in cloudwatch:
http://www.slf4j.org/codes.html#noProviders
what finally worked for me was adding the following dependencies into my own build.gradle:
implementation("org.slf4j:slf4j-api:1.8.0-beta2")
implementation("ch.qos.logback:logback-classic:1.3.0-alpha4") {
exclude("com.sun.mail:javax.mail")
}
implementation ("javax.activation:activation:1.1.1")
implementation("org.jlib:jlib-awslambda-logback:1.0.0")
i'm using java11 and gradle 5.6.4 (kotlin syntax). for some reason the runtimeOnly
stuff wasnt working and i also wasn't getting transitive dependencies properly. the javax.activation stuff is related to java8+ as far as i could tell.
since i saw you asking in vain for more details on that stackoverflow answer, i thought this might be of interest.
Hi, I took me some time to get pass the No SLF4J issue. with the current config in my build.gradle
// runtime dependencies
implementation 'org.slf4j:slf4j-api:1.8.0-beta4'
implementation ('ch.qos.logback:logback-classic:1.3.0-alpha4') {
exclude group: 'ch.qos.logback', module: 'logback-core'
}
compile 'ch.qos.logback:logback-core:1.3.0-alpha4'
runtimeOnly 'org.jlib:jlib-awslambda-logback:1.0.0
once I got passed that I have the following in my logback.xml:
<appender name="awslambda" class="org.jlib.cloud.aws.lambda.logback.AwsLambdaAppender">
<encoder type="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{ISO8601} <%-36X{AWSRequestId}> [%t] %-5p %c:%L - %m%n</pattern>
</encoder>
</appender>
I've enabled scan and debug on the configuration. Here is the output:
22:40:36,101 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - No 'scanPeriod' specified. Defaulting to 1 minutes
22:40:36,228 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Will scan for changes in [file:/var/task/logback.xml]
22:40:36,228 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeTask scanning period to 1 minutes
22:40:36,281 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
22:40:36,334 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [analytics-appender]
22:40:36,963 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.fieldnames.LogstashFieldNames] for [fieldNames] property
22:40:39,376 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [org.jlib.cloud.aws.lambda.logback.AwsLambdaAppender]
22:40:39,389 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [awslambda]
22:40:39,391 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
22:40:39,784 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [analytics] to DEBUG
22:40:39,784 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [analytics] to false
22:40:39,785 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [analytics-appender] to Logger[analytics]
22:40:39,799 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [lambda.export] to DEBUG
22:40:39,800 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
22:40:39,800 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [awslambda] to Logger[ROOT]
22:40:39,800 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
22:40:39,809 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@25bbf683 - Registering current configuration as safe fallback point
2021-07-13 22:40:42,621 <> INFO org.springframework.context.annotation.AnnotationConfigApplicationContext:581 - Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@3c0a50da: startup date [Tue Jul 13 22:40:42 GMT 2021]; root of context hierarchy
2021-07-13 22:40:45,761 <> INFO org.springframework.beans.factory.support.DefaultListableBeanFactory:818 - Overriding bean definition for bean 'dynamoDBClient' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=DBCommonConfig; factoryMethodName=dynamoDBClient; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [DBCommonConfig.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=exportConfig; factoryMethodName=dynamoDBClient; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [ExportConfig.class]]
2021-07-13 22:40:45,764 <> INFO org.springframework.beans.factory.support.DefaultListableBeanFactory:818 - Overriding bean definition for bean 'mapper' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=DBCommonConfig; factoryMethodName=mapper; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [DBCommonConfig.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=true; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=commonUtilConfig; factoryMethodName=mapper; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [CommonUtilConfig.class]]
2021-07-13 22:40:45,765 <> INFO org.springframework.beans.factory.support.DefaultListableBeanFactory:818 - Overriding bean definition for bean 'sqsClient' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=exportConfig; factoryMethodName=sqsClient; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [ExportConfig.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=commonUtilConfig; factoryMethodName=sqsClient; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [CommonUtilConfig.class]]
2021-07-13 22:41:02,453 < > [main] DEBUG t:32 - ResponseReport handler started
2021-07-13 22:41:04,682 < > [main] INFO t:52 - Started Report Lambda with request: {year=2021, month=05}
2021-07-13 22:41:04,751 < > [main] INFO
Note the last three lines. That is the place holder for the AWSRequestId
I have added the layout in the appender like this
**
<appender name="awslambda" class="org.jlib.cloud.aws.lambda.logback.AwsLambdaAppender">
<encoder type="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.example.app.MaskingPatternLayout">
<maskPattern>(\w+@\w+\.\w+)</maskPattern>
<pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] <%-36X{AWSRequestId}> <%-36X{messageUuid}> %-5level
%logger{10} - %msg%n
</pattern>
</layout>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="awslambda"/>
</root>
**
MaskingPatternLayout class:
package com.example.app;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.spi.ILoggingEvent;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.IntStream;
public class MaskingPatternLayout extends PatternLayout {
private Pattern pattern = null;
public void addMaskPattern(String maskPattern) {
pattern = Pattern.compile(maskPattern, Pattern.MULTILINE);
}
@Override
public String doLayout(ILoggingEvent event) {
String log = super.doLayout(event);
StringBuilder logBuilder = new StringBuilder(log);
Matcher matcher = pattern.matcher(logBuilder);
while (matcher.find()) {
int startIndex = matcher.start();
int endIndex = matcher.end();
IntStream.range(startIndex, endIndex).forEach(i -> logBuilder.setCharAt(i, '*'));
}
return logBuilder.toString();
}
}
Logs when invoking the Lambda Function:
|-ERROR in ch.qos.logback.core.joran.util.PropertySetter@77a567e1 - Could not invoke method setLayout in class ch.qos.logback.classic.encoder.PatternLayoutEncoder with parameter of type MaskingPatternLayout java.lang.reflect.InvocationTargetException
at java.lang.reflect.InvocationTargetException
at at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at at java.lang.reflect.Method.invoke(Method.java:498)
at at ch.qos.logback.core.joran.util.PropertySetter.invokeMethodWithSingleParameterOnThisObject(PropertySetter.java:250)
at at ch.qos.logback.core.joran.util.PropertySetter.setComplexProperty(PropertySetter.java:300)
at at ch.qos.logback.core.joran.action.NestedComplexPropertyIA.end(NestedComplexPropertyIA.java:173)
at at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
at at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
at at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
at at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
at at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:82)
at at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:157)
at at ch.qos.logback.classic.spi.LogbackServiceProvider.initializeLoggerContext(LogbackServiceProvider.java:49)
at at ch.qos.logback.classic.spi.LogbackServiceProvider.initialize(LogbackServiceProvider.java:40)
at at org.slf4j.LoggerFactory.bind(LoggerFactory.java:153)
at at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:141)
at at org.slf4j.LoggerFactory.getProvider(LoggerFactory.java:419)
at at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:405)
at at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:354)
at at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:380)
at at com.example.app.handlers.LoggingHandler.(LoggingHandler.java:13)
at at java.lang.Class.forName0(Native Method)
at at java.lang.Class.forName(Class.java:348)
at at lambdainternal.HandlerInfo.fromString(HandlerInfo.java:33)
at at lambdainternal.AWSLambda.findUserMethods(AWSLambda.java:114)
at at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:241)
at at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:208)
at at lambdainternal.AWSLambda.main(AWSLambda.java:198)
Caused by: java.lang.UnsupportedOperationException: one cannot set the layout of ch.qos.logback.classic.encoder.PatternLayoutEncoder
at at ch.qos.logback.core.pattern.PatternLayoutEncoderBase.setLayout(PatternLayoutEncoderBase.java:62)
at ... 33 common frames omitted
21:17:15,410 |-ERROR in ch.qos.logback.classic.PatternLayout("null") - Empty or null pattern.
21:17:15,410 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:17:15,410 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [awslambda] to Logger[ROOT]
21:17:15,412 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:17:15,413 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@736e9adb - Registering current configuration as safe fallback point
START RequestId: 7f74005b-6462-4c3d-a042-4c35feb3e196 Version: $LATEST
END RequestId: 7f74005b-6462-4c3d-a042-4c35feb3e196
REPORT RequestId: 7f74005b-6462-4c3d-a042-4c35feb3e196 Duration: 458.65 ms Billed Duration: 459 ms Memory Size: 512 MB Max Memory Used: 94 MB Init Duration: 1003.82 ms
I'm wondering if your use of a 1.8 beta version of slf4j is deliberate...
Seems like that Graal compile fails because of classes shall or should not be reachable at compile time and runtime. A cfg entry in in oracle reachabiloity github repo is needed
Failed generating 'cloud-function-aws' after 1m 12s.
Error: Classes that should be initialized at run time got initialized during image building:
org.slf4j.LoggerFactory was unintentionally initialized at build time. To see why org.slf4j.LoggerFactory got initialized use --trace-class-initialization=org.slf4j.LoggerFactory
Error: Classes that should be initialized at run time got initialized during image building:
ch.qos.logback.core.util.Duration was unintentionally initialized at build time. To see why ch.qos.logback.core.util.Duration got initialized use --trace-class-initialization=ch.qos.logback.core.util.Duration
ch.qos.logback.classic.model.ConfigurationModel was unintentionally initialized at build time. To see why ch.qos.logback.classic.model.ConfigurationModel got initialized use --trace-class-initialization=ch.qos.logback.classic.model.ConfigurationModel
To see how the classes got initialized, use --trace-class-initialization=org.slf4j.LoggerFactory,ch.qos.logback.core.util.Duration,ch.qos.logback.classic.model.ConfigurationModel
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1
[INFO] ------------------------------------------------------------------------
We are currently relying on the following versions that are becoming outdated and have some vulnerabilities:
- ch.qos.logback/logback-classic "1.3.0-alpha4"
- org.slf4j/log4j-over-slf4j "1.8.0-beta2"
Couldn't we upgrade those versions ? Not sure how difficult it is to do it in this context
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.