christiankniep / docker-elk Goto Github PK

View Code? Open in Web Editor NEW

105.0 13.0 30.0 2.38 MB

Dockerfile creating ELK services (Elasticsearch/Logstash/Kibana)

License: MIT License

Nginx 16.23% Shell 50.08% JavaScript 33.70%

docker-elk's Introduction

docker-elk

Dockerfile creating ELK services (Elasticsearch/Logstash/Kibana)

Please use at least 2GB of RAM for this one... See #16

It's available on hub.docker.com, just pull it: docker pull qnib/elk

Parts

It will

connects with consul, if available
start sshd
start logstash
start diamond
start StatsD
start elasticsearch
start nginx (kibana3)
start kibana4

How to use kibana3 and kibana4 could be explored within this 'hello world' blog post.

Within QNIBTerminal

To get the most out of it a carbon container might be added, but this will impose the question whether to go even further and distribute all the services.

Known issues

Time mismatch in rsyslog

If you forward syslog from rsyslogd, you might encounter a mismatch between UTC and CET. To fix this use this configuration:

# Provide a propper timeformat to fix the UTC/CET mismatch
$template forward_template,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
*.* @@127.0.0.1:5514;forward_template

docker-elk's People

Contributors

Stargazers

Watchers

docker-elk's Issues

To many layers

Ehllo,
I really like this image. It helped me a lot. But I really think you should try to optimize the dockerfile to reduce the amount of layers. You can start with chaining adjecent commands and try to COPY/ADD several files/folders in one command.
read more hints on https://blog.tutum.co/2014/10/22/how-to-optimize-your-dockerfile/

updating kibana views

Hey @ChristianKniep - hopefully quick question. Is there a reason that the default console views in kibana are not externally linked to persist across restarts? I'm at a bit of a sticking point where any work I save gets wiped out on restart of the container (thankfully nothing is in prod, so I'm just messing around at this point). Would it be out of norm to provide the startup with another -v option linking to an external data container to persist views across restarts?

logstash restarting.

Logstash keeps stopping for some reason, ironically I cant find any logs for it.. only the stdout/stderr that says:

elk_1 | 2015-11-29 20:35:07,614 INFO exited: logstash (exit status 1; not expected)
elk_1 | 2015-11-29 20:35:07,836 INFO spawned: 'logstash' with pid 26121
elk_1 | 2015-11-29 20:35:08,862 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
elk_1 | 2015-11-29 20:35:17,974 INFO exited: logstash (exit status 1; not expected)
elk_1 | 2015-11-29 20:35:17,991 INFO spawned: 'logstash' with pid 26222

Running 'service logstash restart' returns:

/etc/init.d/logstash: line 41: .: /etc/default/logstash: is a directory
logstash started.

/etc/init.d/logstash: line 41:
[ -r /etc/default/$name ] && . /etc/default/$name

(name=logstash)

I'm not sure how to deal with it without breaking something else.

I can view logs in kibana, but I'm having trouble loading nginx pattern and I just want to make sure this is not the cause of the problem.

submit to hub

Thanks for creating this! Any chance you would submit this to the docker hub? Would be much more convenient.

Elasticsearch doesn't come up (not enough memory in VM)

/var/log/supervisor/elasticsearch.log is empty, so not really sure how to troubleshoot this, I ran the command in the supervisor config for ES manually and it seemed to work fine?

[root@elk supervisor]# cat supervisord.log
2015-10-14 23:25:14,387 CRIT Supervisor running as root (no user in config file)
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/diamond.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/statsd.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/nginx.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/kibana.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/elasticsearch.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/logstash_watchdog.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/logstash.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/watchpsutil.ini" during parsing
2015-10-14 23:25:14,387 WARN Included extra file "/etc/supervisord.d/consul.ini" during parsing
2015-10-14 23:25:14,388 WARN Included extra file "/etc/supervisord.d/rsyslog.ini" during parsing
2015-10-14 23:25:14,418 INFO RPC interface 'supervisor' initialized
2015-10-14 23:25:14,419 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-10-14 23:25:14,419 INFO supervisord started with pid 12
2015-10-14 23:25:15,422 INFO spawned: 'elasticsearch' with pid 15
2015-10-14 23:25:15,424 INFO spawned: 'logstash_watchdog' with pid 16
2015-10-14 23:25:15,428 INFO spawned: 'statsd' with pid 17
2015-10-14 23:25:15,430 INFO spawned: 'diamond' with pid 18
2015-10-14 23:25:15,437 INFO spawned: 'consul' with pid 19
2015-10-14 23:25:15,451 INFO spawned: 'kibana' with pid 23
2015-10-14 23:25:15,460 INFO spawned: 'watchpsutil' with pid 26
2015-10-14 23:25:15,462 INFO spawned: 'nginx' with pid 27
2015-10-14 23:25:15,479 INFO spawned: 'rsyslog' with pid 32
2015-10-14 23:25:15,565 INFO exited: elasticsearch (exit status 1; not expected)
2015-10-14 23:25:16,474 INFO success: logstash_watchdog entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,474 INFO success: statsd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,474 INFO success: diamond entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,475 INFO success: kibana entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,475 INFO success: watchpsutil entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,475 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,490 INFO success: rsyslog entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-10-14 23:25:16,701 INFO spawned: 'elasticsearch' with pid 288
2015-10-14 23:25:16,722 INFO spawned: 'logstash' with pid 290
2015-10-14 23:25:16,771 INFO exited: elasticsearch (exit status 1; not expected)

Support for Kibana 4

Now that Kibana 4 has been released it would be great to have the option to use it!

port 5514 not listening

Hello @ChristianKniep -

Ran into an issue I'm having trouble unwinding:

I'm unable to send anything over 5514 to a new docker host I have built out- and when I spawn the elk container, it does not look like anything is starting up a port to listen on 5514:

[root@fa503a62eeac nginx]# supervisor_daemonize.sh
Error: Another program is already listening on a port that one of our HTTP servers is configured to use.  Shut this program down first before starting supervisord.
For help, use /usr/bin/supervisord -h
# supervisorctl status
consul                           RUNNING   pid 13, uptime 0:02:03
diamond                          RUNNING   pid 12, uptime 0:02:03
elasticsearch                    RUNNING   pid 8, uptime 0:02:03
logstash                         RUNNING   pid 1838, uptime 0:00:03
logstash_watchdog                RUNNING   pid 9, uptime 0:02:03
nginx                            RUNNING   pid 16, uptime 0:02:03
sshd                             RUNNING   pid 11, uptime 0:02:03
statsd                           RUNNING   pid 10, uptime 0:02:03
syslog-ng                        RUNNING   pid 14, uptime 0:02:03

[root@fa503a62eeac nginx]# lsof -i -n | egrep 'COMMAND|LISTEN'
COMMAND   PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java        8  root  234u  IPv6 393279      0t0  TCP *:vrace (LISTEN)
java        8  root  363u  IPv6 395313      0t0  TCP *:wap-wsp (LISTEN)
sshd       11  root    3u  IPv4 367506      0t0  TCP *:ssh (LISTEN)
sshd       11  root    4u  IPv6 367508      0t0  TCP *:ssh (LISTEN)
statsd     20  root   10u  IPv4 393259      0t0  TCP *:8126 (LISTEN)
nginx      25  root    8u  IPv4 388535      0t0  TCP 127.0.0.1:webcache (LISTEN)
nginx      25  root    9u  IPv4 388536      0t0  TCP *:http (LISTEN)
syslog-ng  42  root    9u  IPv4 388531      0t0  TCP *:shell (LISTEN)
consul    132  root    3u  IPv6 367517      0t0  TCP *:tmi (LISTEN)
consul    132  root   12u  IPv6 367521      0t0  TCP *:amberon (LISTEN)
consul    132  root   15u  IPv6 367523      0t0  TCP *:8302 (LISTEN)
consul    132  root   17u  IPv6 367525      0t0  TCP *:cvd (LISTEN)
consul    132  root   18u  IPv6 367526      0t0  TCP *:fmtp (LISTEN)
consul    132  root   24u  IPv6 388540      0t0  TCP *:domain (LISTEN)
nginx     140 nginx    8u  IPv4 388535      0t0  TCP 127.0.0.1:webcache (LISTEN)
nginx     140 nginx    9u  IPv4 388536      0t0  TCP *:http (LISTEN)

On another host that is working:

[root@243c7789ba73 opt]# lsof -i -n | egrep 'COMMAND|LISTEN'
COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java       9  root  227u  IPv6  24062      0t0  TCP *:vrace (LISTEN)
java       9  root  358u  IPv6  24106      0t0  TCP *:wap-wsp (LISTEN)
python    13  root    6u  IPv4  14848      0t0  TCP *:hsl-storm (LISTEN)
python    13  root    7u  IPv4  14849      0t0  TCP *:newheights (LISTEN)
python    13  root    8u  IPv4  14850      0t0  TCP *:talon-engine (LISTEN)
python    14  root    6u  IPv4  12679      0t0  TCP *:zephyr-clt (LISTEN)
python    14  root    7u  IPv4  12680      0t0  TCP *:zephyr-hm (LISTEN)
python    14  root    8u  IPv4  12681      0t0  TCP *:afs3-prserver (LISTEN)
gunicorn  18  root    5u  IPv4  23926      0t0  TCP 127.0.0.1:distinct (LISTEN)
nginx     22  root    8u  IPv4  15646      0t0  TCP 127.0.0.1:websm (LISTEN)
nginx     22  root    9u  IPv4  15647      0t0  TCP *:webcache (LISTEN)
nginx     22  root   10u  IPv4  15648      0t0  TCP *:http (LISTEN)
python    23  root    6u  IPv4    725      0t0  TCP *:cfinger (LISTEN)
python    23  root    7u  IPv4    726      0t0  TCP *:mailbox (LISTEN)
nginx     24  root    6u  IPv4  17845      0t0  TCP *:ddi-tcp-1 (LISTEN)
java      25  root   26u  IPv6  20049      0t0  TCP *:shell (LISTEN)
java      25  root   37u  IPv6  24113      0t0  TCP *:5514 (LISTEN)
java      25  root  242u  IPv6  25757      0t0  TCP *:9301 (LISTEN)
nginx     55 nginx    6u  IPv4  17845      0t0  TCP *:ddi-tcp-1 (LISTEN)
nginx     56 nginx    8u  IPv4  15646      0t0  TCP 127.0.0.1:websm (LISTEN)
nginx     56 nginx    9u  IPv4  15647      0t0  TCP *:webcache (LISTEN)
nginx     56 nginx   10u  IPv4  15648      0t0  TCP *:http (LISTEN)
gunicorn  99  root    5u  IPv4  23926      0t0  TCP 127.0.0.1:distinct (LISTEN)
gunicorn 107  root    5u  IPv4  23926      0t0  TCP 127.0.0.1:distinct (LISTEN)
sshd     124  root    3u  IPv4  18705      0t0  TCP *:ssh (LISTEN)
sshd     124  root    4u  IPv6  18707      0t0  TCP *:ssh (LISTEN)

Java is opening a port to listen on 5514.

Have you seen this behavior before?

Update README to match the current consul setup; kick out etcd

run issue

When I try to launch docker run, I get that error:

docker@boot2docker:/mnt/sda/var/lib/boot2docker$ docker run -d -h ${NAME} --name ${NAME}
--dns $(docker inspect -format '{{ .NetworkSettings.IPAddress }}' master)
--dns=$(cat /etc/resolv.conf |grep nameserver|head -n1|awk '{print $2}')
-p 9200:9200 -p 9300:9300 -p 8080:80 -p 5514:5514
qnib/elk

Warning: '-format' is deprecated, it will be replaced by '--format' soon. See usage.
Error: No such image or container: master

docker@boot2docker:/mnt/sda/var/lib/boot2docker$ docker logs 67534
2014/06/24 13:28:48 exec: "/bin/sh": stat /bin/sh: no such file or directory

I would apreciate some help.

Customize JVM properties

According to henry1234 it would be nice to temper with the JVM options to optimize the running container. Question is, what should be optimized? :)

kibana wont stay up

Using Kitematic every time I try to start the container I get

2016-03-02 16:40:52,017 INFO exited: kibana (exit status 1; not expected)
2016-03-02 16:40:53,022 INFO spawned: 'kibana' with pid 584
2016-03-02 16:40:54,024 INFO success: kibana entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-03-02 16:40:55,572 INFO exited: kibana (exit status 1; not expected)

any thoughts?

QnibSetup does not find etcd

Hi,
after setting up docker-elk as advised, supervisord.log states

2015-01-28 16:57:48,282 INFO exited: statsd (exit status 1; not expected)
2015-01-28 16:57:48,300 INFO exited: setup (exit status 1; not expected)

setup.log details

Traceback (most recent call last):
  File "/usr/local/bin/qnib-setup.py", line 21, in <module>
    from qnibsetup import QnibConfig, QnibSetup
  File "/usr/lib/python2.7/site-packages/qnibsetup/__init__.py", line 9, in <module>
    from qnibsetup.setup import QnibSetup
  File "/usr/lib/python2.7/site-packages/qnibsetup/setup.py", line 12, in <module>
    import etcd
ImportError: No module named etcd

Any idea how to solve this?
Thanks in advance!

Redirect root-uri to /kibana

As indicated in the PR #4, the root of nginx should be forwarded to /kibana.

Diamond crashes

Actual:
Diamond service crashing:

elk_1  | 2017-02-04 18:00:30,115 INFO success: diamond entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
elk_1  | 2017-02-04 18:00:51,158 INFO exited: diamond (exit status 127; not expected)

with caused by error:
sed: -e expression #1, char 114: unterminated \`s' command /opt/qnib/bin/start_diamond.sh: line 29: diamond: command not found

Which creates a cascading failure of the rest of the stack.

Expected:
Diamond service should stay running and the rest of the stack should come online.

Environment:

Containers: 9
 Running: 8
 Paused: 0
 Stopped: 1
Images: 19
Server Version: 1.13.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 167
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.42 GiB
Name: ubuntu
ID: <redacted>
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

vmstat:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 3  0      0 22202860 389308 7306512    0    0     0     5    2    0  0  0 100  0  0
 0  0      0 22202900 389308 7306572    0    0     0     0 1224 1234  0  1 99  0  0
 0  0      0 22203232 389308 7306572    0    0     0   192 1438 1646  1  0 99  0  0
 0  0      0 22203108 389308 7306572    0    0     0     0 1020 1268  0  0 100  0  0
 0  0      0 22195372 389308 7306580    0    0     0     0 1106 1291  1  0 99  0  0
 0  0      0 22187264 389308 7306556    0    0     0    44 1235 1414  1  0 99  0  0
 0  0      0 22194752 389308 7306556    0    0     0     0 1345 1438  0  0 99  0  0
 1  0      0 22196712 389308 7306568    0    0     0    20 1896 2020  2  1 97  0  0
 1  0      0 22198104 389308 7306560    0    0     0     0 1560 1873  2  1 98  0  0
 0  0      0 22204840 389308 7306460    0    0     0     0 1080 1266  1  0 98  0  0

docker-compose:

version: '2'
services:
  elk:
    image: qnib/elk
    ports:
      - "5601:5601"
      - "9200:9200"
      - "5044:5044"

Chinese comes to unicode ...

When output Chinese, it's unicode type..
how to solve it...
please tell me

nginx not working

Something is wrong about the documentation.