Comments (5)
Each time the tool accesses the log, does it scan/parse the log file EACH time from the beginning? OR from the point it last accessed the log?
The log file is always opened at its end. If you point the exporter to a big log file, it will not parse the entire file, but instead start reading from the end.
What happens when Nginx is very busy and is writing a lot to the log?
The exporter works with a single Goroutine and should not consume more than one CPU core. If Nginx writes faster than the exporter can read, the metrics will lag behind. Apart from that, nothing bad should happen.
Have you encountered performance issues in the past? Were you able to run any sort of performance tests?
I am intending to use this tool in a container. What are the CPU/RAM resources recommended?
I am running this exporter in a production setup that receives about 11k reqs/min. In that setup, the exporter consumes about 2.4% of a CPU core, around 200 MiB of virtual memory and 38 MiB resident memory.
Hope that helps!
from prometheus-nginxlog-exporter.
from prometheus-nginxlog-exporter.
Hi there!
I have performance troubles with this module(1.7.1 version) while using it with Nginx(1.19) in docker-compose. I use Nginx docker image and a module docker image.
The module config file is very simple and works well. I use syslog, as u can see, but also I try the usual log file and it does not change anything.
Nginx config also is simple, also works well.
Prometheus sends GET
requests to module each 120s.
The backend server get about 5-6 RPS from users. Not heavy load.
But module collects all Nginx logs and keep it in RAM and append new, then send all these logs(old and new) to Prometheus, but not delete already send data from his memory.
And after a few minutes I have smth like this:
After few hours\days i have this picture:
What I do wrong? Maybe some mistakes in config or smth like this?
Thanks!
from prometheus-nginxlog-exporter.
@AndreiDrang Looks like you're adding the response time value as label. This will result in the exporter creating a new time series for each request (only grouping those with absolutely identical response_time
and upstream_*_time
values).
Please drop the relabel
rules for response_time
and upstream_*_time
. These are observations (and as such are processed by the exporter automatically as metric values) and make no sense to be exported as label.
from prometheus-nginxlog-exporter.
@martin-helmich Thanks! Everything looks better after relabels delete.
from prometheus-nginxlog-exporter.
Related Issues (20)
- Reading logs from a several servers HOT 1
- Sign the RPM package HOT 2
- Multiple upstream values not supported HOT 4
- Add arm64 binaries/packages HOT 3
- Yet another error while parsing line issue HOT 2
- Missing logs from access.log HOT 1
- no metrics but all nginx log parameters appear HOT 1
- join two labells HOT 1
- RPM package start,Memory continues to grow and never stops
- 404 error at startup HOT 2
- error while parsing $upstream_response_time: value '"0.001"' could not be parsed into float HOT 1
- Add `unmatch` for the `relabel` processor. HOT 2
- Metrics persistence HOT 4
- Unable to parse logs with common log format HOT 2
- Unable to open file /var/log/nginx/access.log: open /var/log/nginx/access.log: permission denied HOT 4
- No metrics prometheus-nginxlog-exporter! HOT 3
- Expose metrics using https
- Missing metrics
- match replacement rule is incorrect.
- not working HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prometheus-nginxlog-exporter.