Comments (10)
I'm afraid it won't work with log4j2-core:2.8.1. Could you try to upgrade Log4j2 to 2.11+?
from log4j2-elasticsearch.
Unfortunately this is not possible, because the version of Minecraft is fixed.
Is there any chance that I could make some changes to make it work
from log4j2-elasticsearch.
I found the problem. The problem is that the classes of Log4J are loaded in the MinecraftServer classloader and the fix does not work with the package so that they are loaded in the plugin classloader. Is there any way around this fix?
from log4j2-elasticsearch.
from log4j2-elasticsearch.
Try the JsonLayout
. It's the basic one from log4j-core.
You won't able to use object pooling though.
Also, different IndexTemplate
may be required to ingest the data.
from log4j2-elasticsearch.
I think I found a solution.
I simply take the layout of the File Appender from the MinecraftServer
<RollingRandomAccessFile name="File" fileName="logs/latest.log" filePattern="logs/%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout>
<LoggerNamePatternSelector defaultPattern="[%d{HH:mm:ss}] [%t/%level]: [%logger] %minecraftFormatting{%msg}{strip}%n">
<!-- Log root, Minecraft, Mojang and Bukkit loggers without prefix -->
<!-- Disable prefix for various plugins that bypass the plugin logger -->
<PatternMatch key=",net.minecraft.,Minecraft,com.mojang.,com.sk89q.,ru.tehkode.,Minecraft.AWE"
pattern="[%d{HH:mm:ss}] [%t/%level]: %minecraftFormatting{%msg}{strip}%n" />
</LoggerNamePatternSelector>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy />
<OnStartupTriggeringPolicy />
</Policies>
<DefaultRolloverStrategy max="1000"/>
</RollingRandomAccessFile>
Now no exception is thrown.
The problem now is that nothing happens.
For example, if I call logger#info, it appears in the log file, but not in elastic. There is also no index for it.
public void onLoad() {
instance = this;
Logger logger = MinecraftServer.LOGGER;
org.apache.logging.log4j.core.Logger coreLogger = (org.apache.logging.log4j.core.Logger)logger;
createLoggerProgrammatically(coreLogger, createElasticsearchAppenderBuilder(coreLogger, false), Configuration::getAsyncLoggerConfigDelegate);
logger.info("test message");
}
public ElasticsearchAppender.Builder createElasticsearchAppenderBuilder(org.apache.logging.log4j.core.Logger coreLogger, boolean messageOnly) {
JestHttpObjectFactory.Builder jestHttpObjectFactoryBuilder;
jestHttpObjectFactoryBuilder = JestHttpObjectFactory.newBuilder();
jestHttpObjectFactoryBuilder.withConnTimeout(1000)
.withReadTimeout(10000)
.withIoThreadCount(8)
.withDefaultMaxTotalConnectionPerRoute(8)
.withMaxTotalConnection(8)
.withMappingType("_doc");
jestHttpObjectFactoryBuilder.withServerUris(hostname);
IndexTemplate indexTemplate = new IndexTemplate.Builder()
.withName("log4j2_test_jest")
.withSource(loadClasspathResource("classpath:indexTemplate.json"))
.build();
BatchDelivery asyncBatchDelivery = new SpigotAsyncBatchDelivery(BATCH_SIZE + ADDITIONAL_BATCH_SIZE, 1000, jestHttpObjectFactoryBuilder.build(), AsyncBatchDelivery.Builder.DEFAULT_FAILOVER_POLICY, indexTemplate);
IndexNameFormatter indexNameFormatter = RollingIndexNameFormatter.newBuilder()
.withIndexName("log4j2_test_jest")
.withPattern("yyyy-MM-dd-HH")
.build();
return ElasticsearchAppender.newBuilder()
.withName("elastic-appender")
.withMessageOnly(messageOnly)
.withBatchDelivery(asyncBatchDelivery)
.withIndexNameFormatter(indexNameFormatter)
.withLayout((AbstractLayout) coreLogger.getContext().getConfiguration().getAppender("File").getLayout())
.withIgnoreExceptions(false);
}
public void createLoggerProgrammatically(org.apache.logging.log4j.core.Logger coreLogger, ElasticsearchAppender.Builder appenderBuilder, Function<Configuration, AsyncLoggerConfigDelegate> delegateSupplier) {
LoggerContext ctx = coreLogger.getContext();
final Configuration config = ctx.getConfiguration();
Appender appender = appenderBuilder.build();
appender.start();
((LifeCycle2)delegateSupplier.apply(config)).start();
config.addAppender(appender);
}
public class SpigotAsyncBatchDelivery extends AsyncBatchDelivery {
public SpigotAsyncBatchDelivery(int batchSize, int deliveryInterval, ClientObjectFactory objectFactory, FailoverPolicy failoverPolicy, IndexTemplate indexTemplate) {
super(batchSize, deliveryInterval, objectFactory, failoverPolicy, indexTemplate);
}
@Override
protected BatchEmitterServiceProvider createBatchEmitterServiceProvider() {
return new SpigotBatchEmitterServiceProvider();
}
}
public class SpigotBatchEmitterServiceProvider extends BatchEmitterServiceProvider{
private final Logger LOG = StatusLogger.getLogger();
@Override
public BatchEmitter createInstance(int batchSize, int deliveryInterval, ClientObjectFactory clientObjectFactory, FailoverPolicy failoverPolicy) {
BatchEmitterFactory factory = new BulkEmitterFactory(); //Service#load not supported
LOG.info("BatchEmitterFactory class found {}", factory.getClass().getName());
if (factory.accepts(clientObjectFactory.getClass())) {
LOG.info("Using {} as BatchEmitterFactoryProvider", factory);
return factory.createInstance(batchSize, deliveryInterval, clientObjectFactory, failoverPolicy);
}
throw new ConfigurationException(String.format("No compatible BatchEmitter implementations for %s found", clientObjectFactory.getClass().getName()));
}
}
I hope you can help me
from log4j2-elasticsearch.
[15:06:06 INFO]: [io.searchbox.client.AbstractJestClient] Setting server pool to a list of 1 servers: [http://localhost:9200]
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Using multi thread/connection supporting pooling connection manager
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Using default GSON instance
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Node Discovery disabled...
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Idle connection reaping disabled...
[15:06:06 WARN]: log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
[15:06:06 WARN]: log4j:WARN Please initialize the log4j system properly.
[15:06:06 WARN]: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2020-01-22 15:06:07,042 I/O dispatcher 1 WARN One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2020-01-22 15:06:07,044 I/O dispatcher 1 WARN Batch of 1 items failed. Redirecting to org.appenders.log4j2.elasticsearch.NoopFailoverPolicy
from log4j2-elasticsearch.
This is the error message that is thrown
{ "took":2, "errors":true, "items":[ { "index":{ "_index":"log4j2_test_jest-2020-01-22-15", "_type":"_doc", "_id":"dXm8zW8BxM6rA-JRNqoN", "status":400, "error":{ "type":"mapper_parsing_exception", "reason":"failed to parse", "caused_by":{ "type":"not_x_content_exception", "reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes" } } } } ] }
log line: " [15:51:03] [Server thread/INFO]: Found new data pack vanilla, loading it automatically"
{ "index_patterns": [ "log4j2*" ], "settings": { "index": { "refresh_interval": "5s" } }, "mappings": { "properties": { "loggerName": { "type": "keyword", "index": false }, "message": { "type": "keyword", "index": false }, "timeMillis": { "type": "date", "format": "epoch_millis" }, "thread": { "type": "keyword", "index": false }, "level": { "type": "keyword", "index": false } } } }
request: {"index":{"_index":"log4j2_test_jest-2020-01-22-16","_type":"_doc"}}
from log4j2-elasticsearch.
You're getting not_x_content_exception
because PatternLayout
doesn't produce a valid JSON.
Have you tried adding Jackson dependencies to the classpath while using JsonLayout
? If that doesn't work other options are:
- creating your own Minecraft server build with all deps included
- writing your own, Gson based layout (I've found Gson embedded in Minecraft uber-jar)
- writing to a file and using Logstash
from log4j2-elasticsearch.
Closed due to lack of activity
from log4j2-elasticsearch.
Related Issues (20)
- Supporting ecs-logging-java HOT 8
- Package naming conventions HOT 11
- How to pass the value from ThreadContext to ValueProperty? HOT 3
- Can not log to elastic using log4j2 <PatternLayout> HOT 11
- Support elasticsearch data_streams HOT 6
- Failed to Load StackTraceElementMixIn HOT 22
- InvalidTypeNameException on invalid mappingType in ES 5 HOT 8
- Adding custom properties, that change during processing HOT 4
- Log4j2 Zero-Day vulnerability (CVE-2021-44228) HOT 2
- Support OpenSearch 2.x and Elasticsearch 8.x HOT 9
- Unable to determine if index already exists HOT 1
- Support for ECSLayout for elasticsearch-ahc and / or elaticsearch-jest in combination with data streams HOT 10
- Is it possible to deseriailze JSON within the message into properties? HOT 2
- Run elastic search appender in specific environment HOT 2
- Logging-Cleaner ERROR Could not create plugin HOT 3
- How do I add mdc attributes? HOT 1
- Could not initialize ChronicleMapRetryFailoverPolicy HOT 2
- ServiceDiscovery ERROR HCServiceDiscovery: Unable to refresh addresses: Cannot invoke "org.appenders.log4j2.elasticsearch.hc.discovery.NodeInfo$PublishAddress.getPublishAddress()" HOT 1
- Using log4j2-elasticsearch-hc -> Does AsyncLogger still not support adding dynamic keyValue pairs in Jackson JSONLayout HOT 5
- IS JacksonModule is implemented in latest version? HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from log4j2-elasticsearch.