apsops / filebeat-kubernetes Goto Github PK
View Code? Open in Web Editor NEWFilebeat container, alternative to fluentd used to ship kubernetes cluster and pod logs
License: MIT License
Filebeat container, alternative to fluentd used to ship kubernetes cluster and pod logs
License: MIT License
Seems the Filebeat config only ships the pod logs from /var/log/containers/*.log
I'm running Kubernetes in Rancher where Kubernetes itself is containerized and logs to the standard /var/lib/docker/containers folder. Now these files are not as nicely named as the symlinked ones in /var/log/containers, but they hold very important information, like the kubelet log.
Is this shipper aiming to solve this problem?
I'm happy to contribute, but don't know what strategy to follow..
Hi
I get the following error when running the container:
tcp.go:26: WARN DNS lookup failure "'logstash.foo": lookup 'logstash.foo: invalid domain name
My yaml file looks like:
- name: LOGSTASH_HOSTS
value: "'logstash.foo:5959'"
The container works correctly if I pass logstash.foo:5959
in filebeat.yml
e.g.
hosts: ["logstash.foo:5959"]
Any ideas?
Hi,
I'm also trying to use filebeat + ELK instead of a default fluentd solution to collect k8s logs, and this repository is really helpful. Thank you!
Could I just ask you why you're mounting the original log stored location /var/lib/docker/containers, not only /var/log/containers with readonly permission? Is it necessary to run this container?
Hello, I started using the container recently, its working great so far. However, its an env variable to add the cluster_name is your are running multiple clusters. However, I'm unable to see that field after shipping our logs through our logging pipeline. We are using shipping to logstash 5.2.2
. This is not a big deal for us at the moment, but wanted to bring this to your attention.
For k8s-1.6 you need to add:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
to get it running on the masters since they are launched with taints.
Today it shows the pod name as host.
The FluentD daemonset (https://github.com/fluent/fluentd-kubernetes-daemonset) follows a better practice, it shows the physical name of the node the log is coming from.
I'm thinking that the physical name would be more helpful in a troubleshooting situation. Patterns pointing to node failure would be easier to identify. Plus showing the pod name is kinda redundant. It's showing up in other fields as well.
My filebeat DaemonSet was not able to read any container log files using the provided yaml. I kept getting error: stat (logfile-path.log) no such file or directory.
To fix the issue I had to also mount the /var/log/pods directory into the container.
I wasn't able to track down the exact change log, but I believe this is due to changes with how the log files are mounted in kubernetes 1.6.
My working daemonset:
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: filebeat
labels:
app: filebeat
spec:
template:
metadata:
labels:
app: filebeat
name: filebeat
spec:
containers:
- name: filebeat
image: apsops/filebeat-kubernetes:v0.3
resources:
limits:
cpu: 50m
memory: 50Mi
env:
- name: LOGSTASH_HOSTS
value: my-host:my-port
- name: LOG_LEVEL
value: debug
volumeMounts:
- name: varlog
mountPath: /var/log/containers
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlogpods
mountPath: /var/log/pods
readOnly: true
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log/containers
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlogpods
hostPath:
path: /var/log/pods
PS. Thanks for putthing this project together, it saved my quite a bit of time.
In my container, the variables in /etc/filebeat/filebeat.yml wouldn't replaced, how does the process works?
Should probably use ConfigMaps to customize filebeat.yml
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.