Giter VIP home page Giter VIP logo

Comments (10)

rfoltyns avatar rfoltyns commented on May 28, 2024

I'm afraid it won't work with log4j2-core:2.8.1. Could you try to upgrade Log4j2 to 2.11+?

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

Unfortunately this is not possible, because the version of Minecraft is fixed.
Is there any chance that I could make some changes to make it work

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

I found the problem. The problem is that the classes of Log4J are loaded in the MinecraftServer classloader and the fix does not work with the package so that they are loaded in the plugin classloader. Is there any way around this fix?

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

from log4j2-elasticsearch.

rfoltyns avatar rfoltyns commented on May 28, 2024

Try the JsonLayout. It's the basic one from log4j-core.
You won't able to use object pooling though.
Also, different IndexTemplate may be required to ingest the data.

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

I think I found a solution.
I simply take the layout of the File Appender from the MinecraftServer

    <RollingRandomAccessFile name="File" fileName="logs/latest.log" filePattern="logs/%d{yyyy-MM-dd}-%i.log.gz">
        <PatternLayout>
            <LoggerNamePatternSelector defaultPattern="[%d{HH:mm:ss}] [%t/%level]: [%logger] %minecraftFormatting{%msg}{strip}%n">
                <!-- Log root, Minecraft, Mojang and Bukkit loggers without prefix -->
                <!-- Disable prefix for various plugins that bypass the plugin logger -->
                <PatternMatch key=",net.minecraft.,Minecraft,com.mojang.,com.sk89q.,ru.tehkode.,Minecraft.AWE"
                              pattern="[%d{HH:mm:ss}] [%t/%level]: %minecraftFormatting{%msg}{strip}%n" />
            </LoggerNamePatternSelector>
        </PatternLayout>
        <Policies>
            <TimeBasedTriggeringPolicy />
            <OnStartupTriggeringPolicy />
        </Policies>
        <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

Now no exception is thrown.
The problem now is that nothing happens.
For example, if I call logger#info, it appears in the log file, but not in elastic. There is also no index for it.

public void onLoad() {
    instance = this;

    Logger logger = MinecraftServer.LOGGER;
    org.apache.logging.log4j.core.Logger coreLogger = (org.apache.logging.log4j.core.Logger)logger;
    createLoggerProgrammatically(coreLogger, createElasticsearchAppenderBuilder(coreLogger, false), Configuration::getAsyncLoggerConfigDelegate);
    logger.info("test message");
}


public  ElasticsearchAppender.Builder createElasticsearchAppenderBuilder(org.apache.logging.log4j.core.Logger coreLogger, boolean messageOnly) {

    JestHttpObjectFactory.Builder jestHttpObjectFactoryBuilder;
    jestHttpObjectFactoryBuilder = JestHttpObjectFactory.newBuilder();

    jestHttpObjectFactoryBuilder.withConnTimeout(1000)
            .withReadTimeout(10000)
            .withIoThreadCount(8)
            .withDefaultMaxTotalConnectionPerRoute(8)
            .withMaxTotalConnection(8)
            .withMappingType("_doc");

    jestHttpObjectFactoryBuilder.withServerUris(hostname);

    IndexTemplate indexTemplate = new IndexTemplate.Builder()
            .withName("log4j2_test_jest")
            .withSource(loadClasspathResource("classpath:indexTemplate.json"))
            .build();

    BatchDelivery asyncBatchDelivery = new SpigotAsyncBatchDelivery(BATCH_SIZE + ADDITIONAL_BATCH_SIZE, 1000, jestHttpObjectFactoryBuilder.build(), AsyncBatchDelivery.Builder.DEFAULT_FAILOVER_POLICY, indexTemplate);

    IndexNameFormatter indexNameFormatter = RollingIndexNameFormatter.newBuilder()
            .withIndexName("log4j2_test_jest")
            .withPattern("yyyy-MM-dd-HH")
            .build();

    return ElasticsearchAppender.newBuilder()
            .withName("elastic-appender")
            .withMessageOnly(messageOnly)
            .withBatchDelivery(asyncBatchDelivery)
            .withIndexNameFormatter(indexNameFormatter)
            .withLayout((AbstractLayout) coreLogger.getContext().getConfiguration().getAppender("File").getLayout())
            .withIgnoreExceptions(false);
}

public void createLoggerProgrammatically(org.apache.logging.log4j.core.Logger coreLogger, ElasticsearchAppender.Builder appenderBuilder, Function<Configuration, AsyncLoggerConfigDelegate> delegateSupplier) {

    LoggerContext ctx = coreLogger.getContext();

    final Configuration config = ctx.getConfiguration();

    Appender appender = appenderBuilder.build();
    appender.start();

    ((LifeCycle2)delegateSupplier.apply(config)).start();

    config.addAppender(appender);
}

public class SpigotAsyncBatchDelivery extends AsyncBatchDelivery {

    public SpigotAsyncBatchDelivery(int batchSize, int deliveryInterval, ClientObjectFactory objectFactory, FailoverPolicy failoverPolicy, IndexTemplate indexTemplate) {
        super(batchSize, deliveryInterval, objectFactory, failoverPolicy, indexTemplate);
    }

    @Override
    protected BatchEmitterServiceProvider createBatchEmitterServiceProvider() {
        return new SpigotBatchEmitterServiceProvider();
    }
}

public class SpigotBatchEmitterServiceProvider extends BatchEmitterServiceProvider{

    private final Logger LOG = StatusLogger.getLogger();

    @Override
    public BatchEmitter createInstance(int batchSize, int deliveryInterval, ClientObjectFactory clientObjectFactory, FailoverPolicy failoverPolicy) {
        BatchEmitterFactory factory = new BulkEmitterFactory(); //Service#load not supported
        LOG.info("BatchEmitterFactory class found {}", factory.getClass().getName());
        if (factory.accepts(clientObjectFactory.getClass())) {
            LOG.info("Using {} as BatchEmitterFactoryProvider", factory);
            return factory.createInstance(batchSize, deliveryInterval, clientObjectFactory, failoverPolicy);
        }
        throw new ConfigurationException(String.format("No compatible BatchEmitter implementations for %s found", clientObjectFactory.getClass().getName()));
    }
}

I hope you can help me

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

[15:06:06 INFO]: [io.searchbox.client.AbstractJestClient] Setting server pool to a list of 1 servers: [http://localhost:9200]
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Using multi thread/connection supporting pooling connection manager
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Using default GSON instance
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Node Discovery disabled...
[15:06:06 INFO]: [io.searchbox.client.JestClientFactory] Idle connection reaping disabled...
[15:06:06 WARN]: log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
[15:06:06 WARN]: log4j:WARN Please initialize the log4j system properly.
[15:06:06 WARN]: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2020-01-22 15:06:07,042 I/O dispatcher 1 WARN One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2020-01-22 15:06:07,044 I/O dispatcher 1 WARN Batch of 1 items failed. Redirecting to org.appenders.log4j2.elasticsearch.NoopFailoverPolicy

from log4j2-elasticsearch.

Foorcee avatar Foorcee commented on May 28, 2024

This is the error message that is thrown

{ "took":2, "errors":true, "items":[ { "index":{ "_index":"log4j2_test_jest-2020-01-22-15", "_type":"_doc", "_id":"dXm8zW8BxM6rA-JRNqoN", "status":400, "error":{ "type":"mapper_parsing_exception", "reason":"failed to parse", "caused_by":{ "type":"not_x_content_exception", "reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes" } } } } ] }
log line: " [15:51:03] [Server thread/INFO]: Found new data pack vanilla, loading it automatically"

{ "index_patterns": [ "log4j2*" ], "settings": { "index": { "refresh_interval": "5s" } }, "mappings": { "properties": { "loggerName": { "type": "keyword", "index": false }, "message": { "type": "keyword", "index": false }, "timeMillis": { "type": "date", "format": "epoch_millis" }, "thread": { "type": "keyword", "index": false }, "level": { "type": "keyword", "index": false } } } }

request: {"index":{"_index":"log4j2_test_jest-2020-01-22-16","_type":"_doc"}}

from log4j2-elasticsearch.

rfoltyns avatar rfoltyns commented on May 28, 2024

You're getting not_x_content_exception because PatternLayout doesn't produce a valid JSON.
Have you tried adding Jackson dependencies to the classpath while using JsonLayout? If that doesn't work other options are:

  • creating your own Minecraft server build with all deps included
  • writing your own, Gson based layout (I've found Gson embedded in Minecraft uber-jar)
  • writing to a file and using Logstash

from log4j2-elasticsearch.

rfoltyns avatar rfoltyns commented on May 28, 2024

Closed due to lack of activity

from log4j2-elasticsearch.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.