Comments (5)
I think you need to provide more details on what you did. You're using a container, did you mount the logs into it?
You're also using the debug container so can shell in and check.
from fluent-bit.
Yes i using two containers(demo application, fluentbit), and mount log file path in both containers
how can i using debug container check, Where can I find documents for reference?
from fluent-bit.
https://kubernetes.io/docs/tasks/debug/debug-application/get-shell-running-container/
from fluent-bit.
Thank you for your response, but unfortunately, I am unable to use that method.
I am using gcp Cloud Run deploy, and Cloud run is a fully managed service and does not allow exec into the container.
below is my test config
Sample app setting
Sample app dockerfile
FROM ubuntu:latest
# Install Python 3
RUN apt-get update && \
apt-get install -y python3 && \
rm -rf /var/lib/apt/lists/*
# Expose port 8080
EXPOSE 8080
# Create a directory for the file
RUN mkdir -p /usr/src/app
RUN mkdir -p /logs
# Copy your script into the container
COPY entrypoint.sh /usr/src/app/
# Set the working directory
WORKDIR /usr/src/app
# Make the script executable
RUN chmod +x entrypoint.sh
# Run the entrypoint script
CMD ["./entrypoint.sh"]
Sample app entrypoint.sh
#!/bin/bash
# Start a simple Python HTTP server on port 8080 and log requests to a file
python3 -u -m http.server 8080 2> /logs/access.log
Sample app build and push to gcp project
docker build -t gcr.io/{project}/demo-app:v1
docker push gcr.io/{project}/demo-app:v1
fluentbit setting
fluentbit dockerfile
FROM fluent/fluent-bit:2.2.0-debug
RUN mkdir -p /logs
COPY fluent-bit.conf /fluent-bit/etc/fluent-bit.conf
fluentbit conf
[Service]
log_level trace
flush 10
[INPUT]
Name tail
Tag samplapp
Path /logs/*.log
Refresh_Interval 90
Read_from_Head true
[OUTPUT]
Name stdout
Match samplapp
fluentbit build and push to gcp project
docker build -t gcr.io/{project}/fluentbit-test:v1
docker push gcr.io/{project}/fluentbit-test:v1
gcp cloud run yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: demo-log
labels:
cloud.googleapis.com/location: asia-east1
annotations:
run.googleapis.com/launch-stage: BETA
run.googleapis.com/ingress: internal
run.googleapis.com/ingress-status: internal
run.googleapis.com/minScale: '1'
run.googleapis.com/max-surge: '0'
spec:
template:
spec:
containers:
- name: demo-1
image: gcr.io/{project}/demo-app:v1
ports:
- name: http1
containerPort: 8080
resources:
limits:
cpu: 1000m
memory: 512Mi
volumeMounts:
- name: in-memory-1
mountPath: /logs
- name: demo-2
image: gcr.io/{project}/fluentbit-test:v1
resources:
limits:
cpu: 1000m
memory: 512Mi
volumeMounts:
- name: in-memory-1
mountPath: /logs
volumes:
- name: in-memory-1
emptyDir:
medium: Memory
sizeLimit: 128Mi
gcp cloud run deploy service
gcloud run services replace gcloud-demo.yaml
and log will appear
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1620 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1694 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1009 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [debug] [input:tail:tail.0] scan_blog add(): dismissed: /logs/access.log, inode 2
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [debug] [input:tail:tail.0] 0 new files found on path '/logs/*.log'
from fluent-bit.
Is there any other information I need to provide about this issue, or are there any updates?
from fluent-bit.
Related Issues (20)
- Bulk size in output es module
- Datadog output setup proxy with user:password HOT 2
- Frequent Restarting of Fluentbit Pods Due to OOMKilled Error HOT 1
- Metrics scrapped using input prometheus scrape doesn't show when using filter stdout HOT 7
- Adding OpenTelemetry attributes to logs forwarded to output Loki HOT 2
- Frequent 'kubelet upstream connection errors' during startup
- Adding "\n" while using multiline.parser with Tail HOT 3
- Does cloudwatch plugin support LogGroup tag? HOT 3
- Hot Reload _sometimes_ does not work HOT 1
- Promethues Metric Buckets Exported Incorrect
- es output fails forever on Windows if remote end closes the connection first
- GELF output plugin doesn't send the tag
- fluentbit_filter_drop_records_total metric is increasing when using multiline filter HOT 1
- Tail + Multi-line parser consumes more than 100% CPU
- time key is missing from the sent data when using the splunk output with Splunk_Send_Raw on
- splunk output with splunk_send_raw on is using the wrong endpoint HOT 2
- Tail plugin incorrectly removing entries from the database file during startup
- Excessive and duplicate writing by prometheus remote writer
- Fluent bit fails to send logs collected by opentelemetry input source to opensearch output
- Increase metric name length to avoid truncating the metric name
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-bit.