willdurand / docker-elk Goto Github PK
View Code? Open in Web Editor NEW๐ณ Creating an ELK stack could not be easier.
Home Page: https://hub.docker.com/r/willdurand/elk/
๐ณ Creating an ELK stack could not be easier.
Home Page: https://hub.docker.com/r/willdurand/elk/
When I try to run a container based off this image, I get the following:
vagrant@dockerhello:~$ docker run -p 8200:80 -p 5000:5000 -p 9200:9200 elk
2015-09-16 17:52:12,185 CRIT Supervisor running as root (no user in config file)
2015-09-16 17:52:12,185 WARN Included extra file "/etc/supervisor/conf.d/kibana.conf" during parsing
2015-09-16 17:52:12,185 WARN Included extra file "/etc/supervisor/conf.d/logstash.conf" during parsing
2015-09-16 17:52:12,186 WARN Included extra file "/etc/supervisor/conf.d/elasticsearch.conf" during parsing
2015-09-16 17:52:12,199 INFO RPC interface 'supervisor' initialized
2015-09-16 17:52:12,200 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-09-16 17:52:12,200 INFO supervisord started with pid 1
2015-09-16 17:52:13,204 INFO spawned: 'elasticsearch' with pid 9
2015-09-16 17:52:13,205 INFO spawned: 'logstash' with pid 10
2015-09-16 17:52:13,209 INFO spawned: 'kibana' with pid 11
2015-09-16 17:52:14,367 INFO success: elasticsearch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-09-16 17:52:14,368 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-09-16 17:52:14,369 INFO success: kibana entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-09-16 17:52:29,389 INFO exited: elasticsearch (terminated by SIGKILL; not expected)
2015-09-16 17:52:29,425 INFO spawned: 'elasticsearch' with pid 63
2015-09-16 17:52:29,848 INFO exited: elasticsearch (exit status 1; not expected)
2015-09-16 17:52:30,855 INFO spawned: 'elasticsearch' with pid 72
2015-09-16 17:52:30,983 INFO exited: elasticsearch (exit status 1; not expected)
2015-09-16 17:52:33,873 INFO spawned: 'elasticsearch' with pid 81
2015-09-16 17:52:33,974 INFO exited: elasticsearch (exit status 1; not expected)
2015-09-16 17:52:37,879 INFO spawned: 'elasticsearch' with pid 92
2015-09-16 17:52:37,983 INFO exited: elasticsearch (exit status 1; not expected)
2015-09-16 17:52:38,984 INFO gave up: elasticsearch entered FATAL state, too many start retries too quickly
# I killed the process here because elasticsearch had died.
^C2015-09-16 17:52:43,958 WARN received SIGINT indicating exit request
2015-09-16 17:52:43,963 INFO waiting for logstash, kibana to die
2015-09-16 17:52:43,967 INFO stopped: kibana (exit status 143)
2015-09-16 17:52:44,012 INFO stopped: logstash (exit status 0)
Note that the "elk" image is an image I made with this Dockerfile:
FROM willdurand/elk
COPY logstash-logspout.conf /etc/logstash/logstash-logspout.conf
Any idea why I might be seeing this?
This is a nice-to-have that the official library/logstash image has but your elk image does not. If I "docker run" this image with /bin/bash
command for some interactive experimentation, the default root shell's PATH
does not include /opt/logstash/bin
so just typing logstash
results in command not found error. In the library/logstash
this is set up for you nicely.
The configuration script leaves a space between the ip host and port in /etc/filebeat/filebeat.yml
output:
logstash:
enabled: true
hosts:
- XX.XX.XX.XX: 5044
#Correct without space
output:
logstash:
enabled: true
hosts:
- XX.XX.XX.XX:5044
line 5: cannot unmarshal !!map into string. Exiting.
Loading config file error: YAML config parsing failed on /etc/filebeat/filebeat.yml: yaml: unmarshal errors:
line 5: cannot unmarshal !!map into string. Exiting.
Regards
I stuck at the first step of configuring Kibana, which is Configure an index pattern
.
I checked the two boxes Index contains time-based events
and Use event times to create index names
. I used the default index name or pattern [logstash-]YYYY.MM.DD
. But Kibana says Unable to fetch mapping. Do you have indices matching the pattern?
.
I'm not able to continue.
Given I run 'docker-compose up -d'
When I open kibana on localhost:8080
Then I see error 'Unable to connect to Elasticsearch at http://localhost:9200 .'
I have tried to curl localhost:9200 after docker-compose exec elk bash. But it not responding.
Anything I am missing? Going to see if I can find the yml config for elastic
Sample docker compose
elk:
image: willdurand/elk:kibana-4.5.4
ports:
- "8080:80"
volumes:
- ./config:/etc/logstash
volumes_from:
- dataelk
dataelk:
image: busybox
volumes:
- /data
When I start the container I notice the log messages of the processes being started in the elk container. However they seem to take a while to actually start processing. E.g. logstash misses many of the initial log messsages produced by other containers. The Kibana site is also available only after some time. Is there some initialization processing going on before data can be processed?
Your logstash configuration SHOULD contain the following input definition:
Does the provided config version complement the default one? Or do you have to include the output
section in my custom logstash.conf
as well:
output {
elasticsearch {
host => elasticsearch
}
}
More generally, it would have been helpful if the README.md provided complete instructions to one complete example of this container being used.
Hi.
First, thanks for this github repo. It's really cool to be able to get started quickly with all that stuff without making some complicated networks and multiple containers with docker.
I have some troubles tho. I pulled 5.X
docker images, and logstash refuses to start due to the config file he can't find (whereas the conf file is here in /etc/logstash
) :
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /etc/logstash/log4j2.properties. Using default config which logs to console
ERROR: No configuration file was specified. Perhaps you forgot to provide the '-f yourlogstash.conf' flag?
usage:
bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
bin/logstash --help
2017-05-07 00:32:19,592 INFO exited: logstash (exit status 1; not expected)
My command line :
docker run --name elk -d -p 4242:80 -v E:/docker/logstash:/etc/logstash willdurand/elk:kibana-5.x
The config file :
input {
lumberjack {
port => 5043
}
}
output {
elasticsearch {
host => "127.0.0.1"
cluster => "logstash"
embedded => false
}
}
I can see the config file by making a ls /etc/logstash
into the container. Even a chmod 777
on the file did not change anything. Still no way to make logstash working. It's like he can't see the file while I'm seeing it on the console (attached to the container as root).
Am I missing something ? Sorry for the inconvenience if this is the case, but I really can't find why anywhere.
Thanks.
Hi,
I've got the following error when running 'docker build .' on Step 5.
Step 5 : RUN apt-key adv --keyserver pool.sks-keyservers.net --recv-keys 46095ACC8548582C1A2699A9D27D666CD88E42B4 && if ! grep "elasticsearch" /etc/apt/sources.list; then echo "deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main" >> /etc/apt/sources.list;fi && if ! grep "logstash" /etc/apt/sources.list; then echo "deb http://packages.elasticsearch.org/logstash/1.4/debian stable main" >> /etc/apt/sources.list;fi && apt-get update
---> Running in 5a6cff0bfe61
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.eNDWVDUFrg --no-auto-check-trustdb --trust-model always --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-stable.gpg --keyserver pool.sks-keyservers.net --recv-keys 46095ACC8548582C1A2699A9D27D666CD88E42B4
gpg: requesting key D88E42B4 from hkp server pool.sks-keyservers.net
?: pool.sks-keyservers.net: Host not found
gpgkeys: HTTP fetch error 7: couldn't connect: Connection timed out
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
The command '/bin/sh -c apt-key adv --keyserver pool.sks-keyservers.net --recv-keys 46095ACC8548582C1A2699A9D27D666CD88E42B4 && if ! grep "elasticsearch" /etc/apt/sources.list; then echo "deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main" >> /etc/apt/sources.list;fi && if ! grep "logstash" /etc/apt/sources.list; then echo "deb http://packages.elasticsearch.org/logstash/1.4/debian stable main" >> /etc/apt/sources.list;fi && apt-get update' returned a non-zero code: 2
And then the build stop... What should I do?
Your help will grandly appreciated.
Regards,
Rocky
When running the elk container, I see a continuous stream of the following:
elk_1 | 2015-05-20 19:45:05,570 INFO spawned: 'logstash' with pid 140
elk_1 | 2015-05-20 19:45:06,572 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
elk_1 | 2015-05-20 19:45:17,404 INFO exited: logstash (exit status 1; not expected)
elk_1 | 2015-05-20 19:45:18,409 INFO spawned: 'logstash' with pid 172
Its seems that continuously restarts. I am not quite sure of the cause, and the logs are not tremendously informative so far.
What is the minimum configuration I need to use this image?
I'm trying to send logs to logstash by using a custom winston transport (https://github.com/jaakkos/winston-logstash).
In the README I read "Your logstash configuration directory MUST contain at least one logstash configuration file" but by configuration file you mean the one in which I specify input, output, filter
?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.