I've been trying to set up Seq in our Kubernetes cluster with GELF input coming from Fluentbit based on this documentation. I have deployed Seq using the Helm chart as described in the documentation. While setting up Fluentbit I've seen that our cluster isn't using the default DNS name of *.cluster.local
but is using a different name instead, so I had to configure Fluentbit slightly different. This could be because our cluster has been rolled out using Rancher.
After fixing that though, I still didn't see any logs. So I had a look at the logs of the seq-gelf
container inside my Seq pod and I saw the following error message:
Failed to send an event batch
System.Net.Http.HttpRequestException: Name or service not known
---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at SeqCli.Ingestion.LogShipper.SendBatchAsync(SeqConnection connection, String apiKey, IReadOnlyCollection`1 batch, Boolean logSendFailures) in /home/appveyor/projects/seqcli/src/SeqCli/Ingestion/LogShipper.cs:line 155
at SeqCli.Ingestion.LogShipper.ShipEvents(SeqConnection connection, String apiKey, ILogEventReader reader, InvalidDataHandling invalidDataHandling, SendFailureHandling sendFailureHandling, Func`2 filter) in /home/appveyor/projects/seqcli/src/SeqCli/Ingestion/LogShipper.cs:line 55
Unfortunately there isn't a way to override this from the Helm chart. I had to manually patch the deployment to change the value of that environment variable. Interestingly I don't think it needs to be a FQDN at all, since the seq-gelf
container is running within the same pod, so it can just use localhost
and it will work. I've manually made that change on my cluster and now I'm seeing logs coming into Seq.