Comments (8)
ShankarSumanth
commented on August 16, 2024
9
Ok finally solved it for MAC. Somehow the configuration
STEP 1: Config file
Create a config file test.conf and put in the below contents
[INPUT]
Name Forward
Port 24224
[OUTPUT]
Name stdout
Match *
Step 2: Start the docker container
Use the below command( and ajust accordingly) to start the fluent bit container.
docker run -it -p 24224:24224 -v $(pwd)/test.conf:/fluent-bit/etc/fluent-bit.conf fluent/fluent-bit:latest
Step 3: Test your configuration
Now in a new terminal start a new docker container and set the log-driver to fluentd
docker run --log-driver=fluentd -t ubuntu echo "Testing a log message".
You should be able to see the output from the docker container, but now I have to figure out why the output message shows up in a wrong format.
For eg:
[0] 72ba16e02658: [1544998108.000000000, {"container_id"=>"72ba16e02658ffb44367972db7fc11c9e8079b0e341152bb9bbf08f6d32e9f17", "container_name"=>"/hungry_hofstadter", "source"=>"stdout", "log"=>"Testing a log "}]sage
Update:
You can of course use the default config provided by fluent-bit but for some reason the config has to be passed as a volume mount as shown in the first step.
from fluent-bit-docker-image.
edsiper
commented on August 16, 2024
1
it's working here:
do you see any difference if you force the IP to 127.0.0.1 ?
from fluent-bit-docker-image.
StudioEtrange
commented on August 16, 2024
1
trying specify 127.0.0.1 for fluentbit only
- not specifying any fluentd driver address
- specifying 127.0.0.1 for mapping
docker run --rm --name fluentbit -p 127.0.0.1:24224:24224 fluent/fluent-bit:1.3.7 /fluent-bit/bin/fluent-bit -i forward://0.0.0.0:24224 -o stdout
Fluent Bit v1.3.7
Copyright (C) Treasure Data
[2020/02/17 02:19:38] [ info] [storage] initializing...
[2020/02/17 02:19:38] [ info] [storage] in-memory
[2020/02/17 02:19:38] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/02/17 02:19:38] [ info] [engine] started (pid=1)
[2020/02/17 02:19:38] [ info] [in_fw] binding 0.0.0.0:24224
[2020/02/17 02:19:38] [ info] [sp] stream processor started
$ docker run --rm --log-driver=fluentd ubuntu echo "Testing a log message"
Testing a log message
trying specify 127.0.0.1 for fluentbit and host ip for driver : generate another error
- specifying 127.0.0.1 for mapping
- specifying HOST_IP (first of the three host address) for fluentd driver address
docker run --rm --name fluentbit -p 127.0.0.1:24224:24224 fluent/fluent-bit:1.3.7 /fluent-bit/bin/fluent-bit -i forward://0.0.0.0:24224 -o stdout
Fluent Bit v1.3.7
Copyright (C) Treasure Data
[2020/02/17 02:05:00] [ info] [storage] initializing...
[2020/02/17 02:05:00] [ info] [storage] in-memory
[2020/02/17 02:05:00] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/02/17 02:05:00] [ info] [engine] started (pid=1)
[2020/02/17 02:05:00] [ info] [in_fw] binding 0.0.0.0:24224
[2020/02/17 02:05:00] [ info] [sp] stream processor started
HOST_IP=$(hostname -I | awk '{split($1, a, " "); print a[1]}')
$ docker run -t --rm --log-driver=fluentd --log-opt fluentd-address=${HOST_IP}:24224 ubuntu echo "Testing a log message"
docker: Error response from daemon: failed to initialize logging driver: dial tcp 192.168.0.50:24224: getsockopt: connection refused.
Ok, when launching fluentbit docker, with 127.0.0.1, the log driver can't connect. I have several IP on my host, so I dont know which fluentd-address use
$ hostname -i
192.168.0.50 192.168.1.50 192.168.2.50
trying specify first hostname IP (192.168.0.50) for fluentbit and driver : error
- specifying one of the host IP for mapping
- specifying one of the host IP for fluentd driver address
docker run --rm --name fluentbit -p 192.168.0.50:24224:24224 fluent/fluent-bit:1.3.7 /fluent-bit/bin/fluent-bit -i forward://0.0.0.0:24224 -o stdout
docker run --rm --log-driver=fluentd --log-opt fluentd-address=192.168.0.50:24224 ubuntu echo "Testing a log message"
docker: Error response from daemon: failed to initialize logging driver: dial tcp 192.168.0.50:24224: i/o timeout.
attach to host net : it works
- not using -t (using -t trunk log)
- attach fluentbit to host network
This works :
$ docker run --rm --name fluentbit --network host -ti fluent/fluent-bit:1.3.7 /fluent-bit/bin/fluent-bit -i forward://0.0.0.0:24224 -o stdout
Fluent Bit v1.3.7
Copyright (C) Treasure Data
[2020/02/17 02:28:41] [ info] [storage] initializing...
[2020/02/17 02:28:41] [ info] [storage] in-memory
[2020/02/17 02:28:41] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/02/17 02:28:41] [ info] [engine] started (pid=1)
[2020/02/17 02:28:41] [ info] [in_fw] binding 0.0.0.0:24224
[2020/02/17 02:28:41] [ info] [sp] stream processor started
[0] 3e2e2292f600: [1581906661.000000000, {"container_id"=>"3e2e2292f600665d70324f030a0a1942bb00b13dc6da85993591a7141e9deb9e", "container_name"=>"/upbeat_bhaskara", "source"=>"stdout", "log"=>"Testing a log message"}]
HOST_IP=$(hostname -I | awk '{split($1, a, " "); print a[1]}')
docker run --rm --log-driver=fluentd --log-opt fluentd-address=${HOST_IP}:24224 ubuntu echo "Testing a log message"
Testing a log message
from fluent-bit-docker-image.
ShankarSumanth
commented on August 16, 2024
Have a similar issue on mac. The fluent-bit docker image outputs logs of cpu.local but was unable to get logs from another container which echo's Testing a log message
Used the same commands provided in the README.
docker run -p 127.0.0.1:24224:24224 fluent/fluent-bit:latest
docker run --log-driver=fluentd -t ubuntu echo "Testing a log message"
from fluent-bit-docker-image.
StudioEtrange
commented on August 16, 2024
On linux
this do not work even using a conf file !
from fluent-bit-docker-image.
edsiper
commented on August 16, 2024
I totally missed this ticket.
@StudioEtrange please elaborate on the issue so I can troubleshoot, steps to reproduce ?
from fluent-bit-docker-image.
StudioEtrange
commented on August 16, 2024
First my env
$ docker version
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:10:22 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:08:52 2018
OS/Arch: linux/amd64
Experimental: false
Secondly, the problem
What i have tested :
HOST_IP=$(hostname -I | awk '{split($1, a, " "); print a[1]}')
docker run --rm --name fluentbit -p 24224:24224 fluent/fluent-bit:1.3.7 /fluent-bit/bin/fluent-bit -i forward://0.0.0.0:24224 -o stdout
docker run --rm --log-driver=fluentd --log-opt fluentd-address=${HOST_IP}:24224 -t ubuntu echo "Testing a log message"
Should ouput log message BUT do not work
Lastly, side notes
My commands are inspired from https://github.com/fluent/fluent-bit-docker-image readme commands which do not work either because default input is cpu not forward. Which is not what readme says
By default, the configuration set a listener on TCP port 24224 through Forward protocol
docker run -p 127.0.0.1:24224:24224 fluent/fluent-bit:latest
docker run --log-driver=fluentd -t ubuntu echo "Testing a log message"
from fluent-bit-docker-image.
edsiper
commented on August 16, 2024
ah, do not use -t
from fluent-bit-docker-image.
Related Issues (20)
- multiarch support for arm/arm64?
- Add k8s-nginx-ingress parser
- fluentbit is not catching an logs with docker (even with the example on the docker hub page) HOT 1
- `master` branch out of date
- Migrate distroless image to Debian 10
- Can we add docker log option to [OUTPUT] by using [MODIFY] or whatever?
- Security patching of fluent bit latest docker image HOT 1
- Kafka out failure in name resolution
- fluent-bit can't run as a different user HOT 2
- How to do health check on docker env. (AWS ECS)? HOT 3
- Can not push to elasticsearch with tls=on and internal certificat
- kube exec does not work for fluent/fluent-bit:1.6.0 HOT 1
- Could not connect my project with Docker and Fluent
- too many open files HOT 4
- Multiple CVEs when installing bin/fluent-bit HOT 1
- Missing tag 1.8.12 and 1.8.13? HOT 2
- 16 issues in a docker image fluent/fluent-bit:1.8.12 HOT 1
- The image has CVE HOT 3
- CVEs in 1.9.3 HOT 1
- What is "source"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-bit-docker-image.