graylog-labs / graylog-plugin-mqtt Goto Github PK

View Code? Open in Web Editor NEW

13.0 18.0 11.0 61 KB

MQTT Input Plugin for Graylog

Home Page: https://www.graylog.org/

Java 100.00%

graylog mqtt mqtt-protocol mqtt-client graylog-plugin iot iot-framework

graylog-plugin-mqtt's Introduction

MQTT Plugin for Graylog

This is an input plugin that allows you to subscribe to an MQTT broker and index all published messages.

Required Graylog version: 2.4.0 and later

Installation

Download the plugin and place the .jar file in your Graylog plugin directory. The plugin directory is the plugins/ folder relative from your graylog-server directory by default and can be configured in your graylog.conf file.

Restart graylog-server and you are done.

Build

This project is using Maven 3 and requires Java 8 or higher.

You can build a plugin (JAR) with mvn package.

DEB and RPM packages can be build with mvn jdeb:jdeb and mvn rpm:rpm respectively.

Plugin Release

We are using the maven release plugin:

$ mvn release:prepare
[...]
$ mvn release:perform

This sets the version numbers, creates a tag and pushes to GitHub. Travis CI will build the release artifacts and upload to GitHub automatically.

graylog-plugin-mqtt's People

Contributors

Stargazers

Watchers

Forkers

tr002196 juey90 eugenegwon hafo821 isabella232 alc6379 bthom2 autoantwort liviob74

graylog-plugin-mqtt's Issues

Messages via input plugin not queryable via search (ex: gl2_source_input)

Hi All

I have this plugin up and running on graylog2 0.91.2

Works fine

plugin loads ok
mqtt messages are received, parsed and processed
Message counter in the right top corner increases when receiving messages
Looking for a message via de Extractor>loadMessage works
- _id: 14eb8cc0-86c8-11e4-b719-080027fcfbc7
- index: graylog2_0
defining a stream detecting and defining an output also works
example output:

org.graylog2.outputs.LoggingOutput - Writing message: mqtt publish received source: 192.168.10.101 | message: {"x":"hoera2"} { timestamp: 46933-08-22T16:10:51.000Z | _mqtt_topic: home/test | _id: 14eb8cc0-86c8-11e4-b719-080027fcfbc7 | gl2_source_input: 54922bbf24ac84d1f4d7d93b | gl2_source_node: 1b1a3d09-4864-4e7f-8e1f-307bb79d1247 | _mqtt_received_timestamp: 1418915463051 | MqttPayload: {"x":"hoera2"} }

Does not work

When going to System>inputs>MQTT input>action>messages from this input it loads nothing.
In the search bar at the top it states: gl2_source_input:54922bbf24ac84d1f4d7d93b
which is obviously ok comparing the value with the logged output above.

It also states: 0 messages found

Attempts

I tried 2 attempts. Pushing the message directly to the processBuffer or by calling the processRawMessage of the InputMessage. Both with the same result.

Thanks in advance for time and effort.
Best regards
Wim

Can not create two inputs for the same MQTT Server

I guess the client id used by the plugin in the same for every source. So the mqtt broker disconnects the connection constantly.

2022-12-09T01:57:28.872+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:28.872+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:28.924+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:28.924+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:28.975+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:28.975+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:29.027+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:29.027+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:29.079+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:29.079+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:29.130+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:29.130+01:00 DEBUG [ClientListener] Disconnected MQTT client
2022-12-09T01:57:29.182+01:00 WARN  [xenqtt] Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2022-12-09T01:57:29.182+01:00 DEBUG [ClientListener] Disconnected MQTT client

If I stop one of the sources the problem disappears.

Crash on launch input in 0.91.3 (and higher)

I'm hearing here that this is also happening on 0.92.1.

Graylog2 0.91.3 launched on an empty ES

dialog

No inputs.
Launch new input: MQTT Input

    Global input: yes
    Title: MQTT-1883
    broker URL: tcp://localhost:1883
    Password: <none>
    Topic names: gl/2
    keepalive: 300
    threadpool: 1
    username: <none>
    connect timeout: 30

    (username 2nd time: <none>)

LauncH: Oops, an error occurred; logged as id 6kgmdhjdl

2014-12-17 10:29:53,744 INFO : org.reflections.Reflections - Reflections took 235 ms to scan 1 urls, producing 8 keys and 18 values 
2014-12-17 10:29:53,940 INFO : org.reflections.Reflections - Reflections took 91 ms to scan 1 urls, producing 5 keys and 19 values 
2014-12-17 10:29:54,435 INFO : org.graylog2.plugin.system.NodeId - Node ID: f4f2f50c-424f-478e-bc87-dec59ae61cfd
2014-12-17 10:29:54,821 INFO : org.graylog2.buffers.OutputBuffer - Initialized OutputBuffer with ring size <1024> and wait strategy <BlockingWaitStrategy>.
2014-12-17 10:29:54,841 INFO : org.graylog2.Main - Graylog2 0.91.3 (edd81aa) starting up. (JRE: Oracle Corporation 1.8.0_25 on Mac OS X 10.10.1)
2014-12-17 10:29:54,873 INFO : org.graylog2.shared.initializers.PeriodicalsService - Starting 14 periodicals ...
2014-12-17 10:29:54,873 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2014-12-17 10:29:54,875 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2014-12-17 10:29:54,876 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.OutputCacheWorkerThread] periodical, running forever.
2014-12-17 10:29:54,876 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2014-12-17 10:29:54,877 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThroughputCounterManagerThread] periodical in [0s], polling every [1s].
2014-12-17 10:29:54,877 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.InputCacheWorkerThread] periodical, running forever.
2014-12-17 10:29:54,878 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.VersionCheckThread] periodical in [0s], polling every [1800s].
2014-12-17 10:29:54,879 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.TelemetryReporterThread] periodical. Not configured to run on this node.
2014-12-17 10:29:54,879 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.DeflectorManagerThread] periodical in [0s], polling every [10s].
2014-12-17 10:29:54,879 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.StreamThroughputCounterManagerThread] periodical in [0s], polling every [1s].
2014-12-17 10:29:54,880 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [0s], polling every [20s].
2014-12-17 10:29:54,880 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling every [60s].
2014-12-17 10:29:54,884 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.DeadLetterThread] periodical, running forever.
2014-12-17 10:29:54,885 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2014-12-17 10:29:55,093 INFO : org.elasticsearch.node - [graylog2-server] version[1.3.4], pid[93057], build[a70f3cc/2014-09-30T09:07:17Z]
2014-12-17 10:29:55,094 INFO : org.elasticsearch.node - [graylog2-server] initializing ...
2014-12-17 10:29:55,099 INFO : org.elasticsearch.plugins - [graylog2-server] loaded [], sites []
2014-12-17 10:29:56,588 INFO : org.drools.compiler.kie.builder.impl.KieRepositoryImpl - KieModule was added:MemoryKieModule[ ReleaseId=org.graylog2:dynamic-rules:0]
2014-12-17 10:29:56,671 INFO : org.drools.compiler.kie.builder.impl.KieRepositoryImpl - Adding KieModule from resource :[ByteArrayResource resource=[B@578ec8fd]
2014-12-17 10:29:56,737 INFO : org.drools.compiler.kie.builder.impl.KieRepositoryImpl - KieModule was added:MemoryKieModule[ ReleaseId=org.graylog2:dynamic-rules:0]
2014-12-17 10:29:56,902 INFO : org.elasticsearch.node - [graylog2-server] initialized
2014-12-17 10:29:56,903 INFO : org.elasticsearch.node - [graylog2-server] starting ...
2014-12-17 10:29:56,961 INFO : org.graylog2.bindings.providers.RulesEngineProvider - No static rules file loaded.
2014-12-17 10:29:56,993 INFO : org.elasticsearch.transport - [graylog2-server] bound_address {inet[/0:0:0:0:0:0:0:0:9350]}, publish_address {inet[/192.168.1.99:9350]}
2014-12-17 10:29:56,998 INFO : org.elasticsearch.discovery - [graylog2-server] elasticsearch/d22VZPr6Q-GIBtq3h7NwqA
2014-12-17 10:29:57,034 INFO : org.graylog2.shared.buffers.ProcessBuffer - Initialized ProcessBuffer with ring size <1024> and wait strategy <BlockingWaitStrategy>.
2014-12-17 10:29:59,885 INFO : org.graylog2.shared.initializers.RestApiService - Adding security context factory: <org.graylog2.security.ShiroSecurityContextFactory@46064cf3>
2014-12-17 10:29:59,893 INFO : org.graylog2.shared.initializers.RestApiService - Started REST API at <http://127.0.0.1:12900/>
2014-12-17 10:30:00,006 WARN : org.elasticsearch.discovery - [graylog2-server] waited for 3s and no initial state was set by the discovery
2014-12-17 10:30:00,006 INFO : org.elasticsearch.node - [graylog2-server] started
2014-12-17 10:30:01,673 INFO : org.elasticsearch.cluster.service - [graylog2-server] detected_master [localhost][eVdZG5HbQV-ipz_J6LrzgQ][tiggr.ww.mens.de][inet[/192.168.1.99:9300]]{master=true}, added {[localhost][eVdZG5HbQV-ipz_J6LrzgQ][tiggr.ww.mens.de][inet[/192.168.1.99:9300]]{master=true},}, reason: zen-disco-receive(from master [[localhost][eVdZG5HbQV-ipz_J6LrzgQ][tiggr.ww.mens.de][inet[/192.168.1.99:9300]]{master=true}])
2014-12-17 10:30:01,984 INFO : org.graylog2.indexer.Deflector - Did not find an deflector alias. Setting one up now.
2014-12-17 10:30:01,986 INFO : org.graylog2.indexer.Deflector - There is no index target to point to. Creating one now.
2014-12-17 10:30:02,000 INFO : org.graylog2.indexer.Deflector - Cycling deflector to next index now.
2014-12-17 10:30:02,001 INFO : org.graylog2.indexer.Deflector - Cycling from <none> to <graylog2_0>
2014-12-17 10:30:02,001 INFO : org.graylog2.indexer.Deflector - Creating index target <graylog2_0>...
2014-12-17 10:30:03,132 INFO : org.graylog2.system.jobs.SystemJobManager - Submitted SystemJob <47719cb0-85cf-11e4-a579-429fd6d54569> [org.graylog2.indexer.ranges.RebuildIndexRangesJob]
2014-12-17 10:30:03,132 INFO : org.graylog2.indexer.Deflector - Done!
2014-12-17 10:30:03,132 INFO : org.graylog2.indexer.Deflector - Pointing deflector to new target index....
2014-12-17 10:30:03,133 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Re-calculating index ranges.
2014-12-17 10:30:03,454 INFO : org.graylog2.indexer.Deflector - Done!
2014-12-17 10:30:03,456 INFO : org.graylog2.shared.initializers.ServiceManagerListener - Services are healthy
2014-12-17 10:30:03,457 INFO : org.graylog2.shared.initializers.InputSetupService - Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2014-12-17 10:30:03,464 INFO : org.graylog2.Main - Services started, startup times in ms: {GroovyShellSetupService [RUNNING]=0, OutputSetupService [RUNNING]=0, BufferSynchronizerService [RUNNING]=0, GelfChunkManagerService [RUNNING]=0, MetricsReporterService [RUNNING]=0, DashboardRegistryService [RUNNING]=1, InputSetupService [RUNNING]=11, PeriodicalsService [RUNNING]=12, ProcessBufferService [RUNNING]=2161, RestApiService [RUNNING]=5020, IndexerSetupService [RUNNING]=8581}
2014-12-17 10:30:03,465 INFO : org.graylog2.Main - Graylog2 up and running.
2014-12-17 10:30:03,843 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Index [graylog2_0] is empty but it is the current deflector target. Inserting dummy index range.
2014-12-17 10:30:03,846 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index ranges for 1 indices. Took 365ms.
2014-12-17 10:30:03,847 INFO : org.graylog2.system.jobs.SystemJobManager - SystemJob <47719cb0-85cf-11e4-a579-429fd6d54569> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 713ms.

2014-12-17 10:31:55,111 INFO : org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Enabling session validation scheduler...
2014-12-17 10:32:05,627 ERROR: org.graylog2.jersey.container.netty.NettyContainer - Uncaught exception during jersey resource handling
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:51)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470)
    at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:146)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:99)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
    at org.jboss.netty.channel.Channels.write(Channels.java:725)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:280)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:121)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
    at org.jboss.netty.channel.Channels.write(Channels.java:704)
    at org.jboss.netty.channel.Channels.write(Channels.java:671)
    at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248)
    at org.graylog2.jersey.container.netty.NettyContainer$NettyResponseWriter$1.write(NettyContainer.java:142)
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
    at java.io.BufferedWriter.flush(BufferedWriter.java:254)
    at org.glassfish.jersey.message.internal.ReaderWriter.writeToAsString(ReaderWriter.java:192)
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:129)
    at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:99)
    at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:59)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:85)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1154)
    at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:621)
    at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:377)
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:367)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028)
    at org.graylog2.jersey.container.netty.NettyContainer.messageReceived(NettyContainer.java:356)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2014-12-17 10:32:12,409 ERROR: org.graylog2.jersey.container.netty.NettyContainer - Uncaught exception during jersey resource handling
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:51)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470)
    at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:146)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:99)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
    at org.jboss.netty.channel.Channels.write(Channels.java:725)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:280)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:121)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
    at org.jboss.netty.channel.Channels.write(Channels.java:704)
    at org.jboss.netty.channel.Channels.write(Channels.java:671)
    at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248)
    at org.graylog2.jersey.container.netty.NettyContainer$NettyResponseWriter$1.write(NettyContainer.java:142)
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
    at java.io.BufferedWriter.flush(BufferedWriter.java:254)
    at org.glassfish.jersey.message.internal.ReaderWriter.writeToAsString(ReaderWriter.java:192)
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:129)
    at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:99)
    at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:59)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:85)
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
    at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1154)
    at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:621)
    at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:377)
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:367)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028)
    at org.graylog2.jersey.container.netty.NettyContainer.messageReceived(NettyContainer.java:356)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2014-12-17 10:34:46,340 ERROR: org.graylog2.jersey.container.netty.NettyContainer - Uncaught exception in transport layer. This is likely a bug, closing channel.
java.lang.NoSuchMethodError: org.graylog2.plugin.configuration.Configuration.getInt(Ljava/lang/String;)J
    at org.graylog2.inputs.mqtt.MQTTInput.checkConfiguration(MQTTInput.java:75)
    at org.graylog2.rest.resources.system.inputs.InputsResource.create(InputsResource.java:147)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028)
    at org.graylog2.jersey.container.netty.NettyContainer.messageReceived(NettyContainer.java:356)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Graylog 4.2.8 in a docker, plugin issue

I am trying to use the plugin 1.1.1 inside docker, it loads and seems configurable, however I can't establish a connection to either local or external mqtt servers. The mosquitto:1883 is the compose name of the service, I have tried tcp:// variants, external ip numbers etc. without any luck.
Any ideas - or is this a dead end?

 2022-04-29 08:56:20,342 ERROR: xenqtt - Failed to connect a client MQTT channel to mosquitto:1883
graylog-graylog-1        | java.nio.channels.UnresolvedAddressException: null
graylog-graylog-1        |      at sun.nio.ch.Net.checkAddress(Net.java:104) ~[?:1.8.0_322]
graylog-graylog-1        |      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:621) ~[?:1.8.0_322]
graylog-graylog-1        |      at net.sf.xenqtt.message.AbstractMqttChannel.<init>(AbstractMqttChannel.java:105) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.MqttClientChannel.<init>(MqttClientChannel.java:43) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.ChannelManagerImpl$NewClientChannelCommand.doExecute(ChannelManagerImpl.java:617) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.AbstractBlockingCommand.execute(AbstractBlockingCommand.java:87) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.ChannelManagerImpl.executeCommands(ChannelManagerImpl.java:441) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.ChannelManagerImpl.doIO(ChannelManagerImpl.java:309) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.ChannelManagerImpl.access$000(ChannelManagerImpl.java:47) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at net.sf.xenqtt.message.ChannelManagerImpl$1.run(ChannelManagerImpl.java:92) [graylog-plugin-mqtt-1.1.1.jar:?]
graylog-graylog-1        |      at java.lang.Thread.run(Thread.java:750) [?:1.8.0_322]
graylog-graylog-1        | 2022-04-29 08:56:20,343 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.mqtt.MQTTGELFInput] input with ID <626ba3c0bc979d2d7d6e8828> misfired. Reason: Command failed: NewClientChannelCommand; ROOT CAUSE: null.

Username field in MQTT launch dialog

is there twice.

Possible version in-compatibility problem?

Using graylog 3.0.0-alpha.1+83bd3b5 (inside docker image) and the graylog-plugin-mqtt v. 1.1.1 I have the error below while subscribe topic "#" of the mqtt broker that I would like to index.

Is it a version in-compatibility problem?
Thanks

graylog_1        | 2018-07-10 21:27:42,619 WARN : org.graylog2.inputs.codecs.GelfCodec - GELF message <1450d2e0-8488-11e8-9ad4-0242ac120005> is missing mandatory "host" field.
graylog_1        | 2018-07-10 21:27:42,623 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Unable to decode raw message RawMessage{id=1450d2e0-8488-11e8-9ad4-0242ac120005, journalOffset=9122, codec=gelf, payloadSize=163, timestamp=2018-07-10T21:27:42.606Z} on input <5b451665adbe1d000169a345>.
graylog_1        | 2018-07-10 21:27:42,623 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing message RawMessage{id=1450d2e0-8488-11e8-9ad4-0242ac120005, journalOffset=9122, codec=gelf, payloadSize=163, timestamp=2018-07-10T21:27:42.606Z}
graylog_1        | java.lang.IllegalArgumentException: GELF message <1450d2e0-8488-11e8-9ad4-0242ac120005> is missing mandatory "short_message" or "message" field.
graylog_1        | 	at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:262) ~[graylog.jar:?]
graylog_1        | 	at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:134) ~[graylog.jar:?]
graylog_1        | 	at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
graylog_1        | 	at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
graylog_1        | 	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
graylog_1        | 	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
graylog_1        | 	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
graylog_1        | 	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
graylog_1        | 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

Feature Request: Support for Client Certificate Authentication

Any Chance this plugin can add support for client certificate based authentication.
Some cloud MQTT services example: AWS IoT only support client certificate authentication.

Thanks

not starting with graylog 5.1

The plugin fails to start with graylog 5.1. It logs the following error:

1) [Guice/MissingImplementation]: No implementation for MQTTGELFInput$Factory was bound.

Requested by:
1  : Graylog2Module.installInput(Graylog2Module.java:233)
      \_ installed by: PluginBindings -> MQTTInputModule

Learn more:
  https://github.com/google/guice/wiki/MISSING_IMPLEMENTATION

2) MessageInput$Descriptor is abstract, not a concrete class.  Unable to create AssistedInject factory.
  while locating MessageInput$Descriptor
  at MessageInput$Factory.getDescriptor(MessageInput.java:1)

It can be reproduced with the following docker-compose file (volume and plugin paths need to be adjusted):

version: '3'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: mongo:6.0
    networks:
      - graylog
    # DB in share for persistence
    volumes:
      - /home/marc_diesse/volumes/mongodb:/data/db
  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
    # data folder in share for persistence
    volumes:
      - /home/marc_diesse/volumes/elasticsearch:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    mem_limit: 1g
    networks:
      - graylog
  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    image: graylog/graylog:5.1
    # journal and config directories in local NFS share for persistence
    volumes:
      - /home/user_name/volumes/graylog_journal:/usr/share/graylog/data/journal
      - /home/user_name/volumes/graylog_plugins/graylog-plugin-mqtt-1.1.1.jar:/usr/share/graylog/plugin/graylog-plugin-mqtt-1.1.1.jar
    environment:
      # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET=graylogpw11111111
      # Password: admin
      - GRAYLOG_ROOT_PASSWORD_SHA2=70832c7aadf276ad91f07dca82c5da7acf4b9d5e74b5e6d3d6963aa11448fd87
      - GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/
    entrypoint: /usr/bin/tini -- wait-for-it elasticsearch:9200 -- /docker-entrypoint.sh
    networks:
      - graylog
    links:
      - mongodb:mongo
      - elasticsearch
    restart: always
    depends_on:
      - mongodb
      - elasticsearch
    ports:
      # Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
# Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
volumes:
  mongo_data:
    driver: local
  es_data:
    driver: local
  graylog_journal:
    driver: local
networks:
  graylog:
    driver: bridge

Can this be solved by building with newer java version?

Plugin stopps working

Hi, I am running against a docker based mosquitto and the plugin stops grabbing new values after approx. 3 hours. Mosquitto continues getting data from the source and another client (MQTT fx) keeps showing these updates so I can exclude mosquitto as the reason for this behaviour. I've tried lowering the keepalive time as well as increasing the timeout but with no avail. Any ideas?
Thanks
Andreas

Plugin stop working after broker restart

I'm testing the MQTT TCP (Raw/Plaintext) input plugin provided after discussion in #6 with discrete success in the last 30 days until it suddenly stop working on 1 August between 6:00 and 7:00.

Apparently the broker was restarted and the plugin wasn't able to reconnect for over a day.
You can see the gap in mqtt acquisition here:

I'm running dockerized version of graylog (ver: v3.0.0-alpha.1+83bd3b5) and mqtt-plugin (ver: 1.2.0-rc.1)

On the grayloag logs we have:

2018-08-01 06:43:01,629 WARN : xenqtt - Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2018-08-01 06:43:01,850 ERROR: xenqtt - Failed to connect MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_171]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_171]
	at net.xenqtt.message.AbstractMqttChannel.finishConnect(AbstractMqttChannel.java:193) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.doConnect(ChannelManagerImpl.java:362) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.doIO(ChannelManagerImpl.java:312) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.access$000(ChannelManagerImpl.java:47) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl$1.run(ChannelManagerImpl.java:92) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
2018-08-01 06:43:01,859 WARN : xenqtt - Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2018-08-01 06:43:01,864 ERROR: xenqtt - Unable to create a new connection.
net.xenqtt.MqttInvocationException: Command failed: NewClientChannelCommand; ROOT CAUSE: Connection refused
	at net.xenqtt.message.AbstractBlockingCommand.await(AbstractBlockingCommand.java:65) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:175) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:166) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:150) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.client.AbstractMqttClient$ClientReconnector.run(AbstractMqttClient.java:791) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
2018-08-01 06:43:02,122 ERROR: xenqtt - Failed to connect MqttClientChannel[localAddress:N/A,remoteAddress:N/A]

It repeated a lot of time until I restarted the graylog input itself:

2018-08-01 07:21:41,218 WARN : xenqtt - Connection to broker lost; scheduling a reconnect attempt for channel: MqttClientChannel[localAddress:N/A,remoteAddress:N/A]
2018-08-01 07:21:41,219 ERROR: xenqtt - Unable to create a new connection.
net.xenqtt.MqttInvocationException: Command failed: NewClientChannelCommand; ROOT CAUSE: Connection refused
	at net.xenqtt.message.AbstractBlockingCommand.await(AbstractBlockingCommand.java:65) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:175) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:166) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.message.ChannelManagerImpl.newClientChannel(ChannelManagerImpl.java:150) ~[graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at net.xenqtt.client.AbstractMqttClient$ClientReconnector.run(AbstractMqttClient.java:791) [graylog-plugin-mqtt-1.2.0-rc.1.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
2018-08-01 12:53:30,147 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Unable to decode raw message RawMessage{id=e3e78821-9589-11e8-b769-0242ac120004, journalOffset=1265664, codec=gelf, payloadSize=316, timestamp=2018-08-01T12:53:30.146Z} on input <5b4dc8952ab79c0001239c56>.
2018-08-01 12:53:30,154 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing message RawMessage{id=e3e78821-9589-11e8-b769-0242ac120004, journalOffset=1265664, codec=gelf, payloadSize=316, timestamp=2018-08-01T12:53:30.146Z}
java.lang.IllegalArgumentException: GELF message <e3e78821-9589-11e8-b769-0242ac120004> has empty mandatory "short_message" field.
	at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:252) ~[graylog.jar:?]
	at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:134) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
2018-08-02 14:39:42,190 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STOPPING
2018-08-02 14:39:42,228 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STOPPED
2018-08-02 14:39:42,255 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now TERMINATED
2018-08-02 14:39:44,534 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:14, serverValue:54406}] to ds139841.mlab.com:39841
2018-08-02 14:39:44,537 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:13, serverValue:54405}] to ds139841.mlab.com:39841
2018-08-02 14:39:44,604 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STARTING
2018-08-02 14:39:44,715 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now RUNNING
2018-08-02 14:40:01,268 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STOPPING
2018-08-02 14:40:01,296 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STOPPED
2018-08-02 14:40:01,329 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now STARTING
2018-08-02 14:40:01,358 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now TERMINATED
2018-08-02 14:40:01,411 INFO : org.graylog2.inputs.InputStateListener - Input [MQTT TCP (Raw/Plaintext)/5b4dc8662ab79c0001239c1b] is now RUNNING
2018-08-02 17:47:23,886 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Unable to decode raw message RawMessage{id=1cd82ac2-967c-11e8-b769-0242ac120004, journalOffset=1496466, codec=gelf, payloadSize=331, timestamp=2018-08-02T17:47:23.884Z} on input <5b4dc8952ab79c0001239c56>.
2018-08-02 17:47:23,897 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Unable to decode raw message RawMessage{id=1cd851d1-967c-11e8-b769-0242ac120004, journalOffset=1496467, codec=gelf, payloadSize=331, timestamp=2018-08-02T17:47:23.885Z} on input <5b4dc8952ab79c0001239c56>.
2018-08-02 17:47:23,899 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing message RawMessage{id=1cd851d1-967c-11e8-b769-0242ac120004, journalOffset=1496467, codec=gelf, payloadSize=331, timestamp=2018-08-02T17:47:23.885Z}
java.lang.IllegalArgumentException: GELF message <1cd851d1-967c-11e8-b769-0242ac120004> has empty mandatory "short_message" field.
	at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:252) ~[graylog.jar:?]
	at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:134) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

#5 could facing a similar issue.

Feature Request: Add mqtt topic as field of recieved message

I'm using this plugin to indexing and searching mqtt messages. I found that having topic information as message field is a really useful information to have for filtering messages.

Can this be implement?

Thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.