Comments (7)
We used to have a watchdog that caught runaway filters in these situations.
We can bring it back - good idea :)
On Thursday, December 10, 2015, Pius [email protected] wrote:
We have users sending in occasional large messages of many Mbs, and if
they have not implemented logic in the pipeline to filter out these
messages, the grok filter will run and peg the CPU threads for a very long
time. It will be helpful to have a default (or configurable) timeout for
grok filters so that it will give up at some point, etc..—
Reply to this email directly or view it on GitHub
#65.
from logstash-filter-grok.
Related: Joni (as of 2.5-ish years ago) supports being interrupted via Thread.interrupt.
from logstash-filter-grok.
Proposal for this:
- Configurable execution timeout. Default 0.5 seconds.
- When execution time expires, we interrupt grok via java Thread.interrupt().
- On interruption, we tag the event _grokexecutiontimeout and continue on to the next filter in the filter worker.
- On interruption, we log a warning indicating that Grok's execution expired and include the following context: the text input, the original pattern being tested, and the execution time.
This may get tricky given the new complexity of the combined filter-and-output worker units as well as having some possibility (we'll need to protect against) of race conditions of grok succeeding vs the timer expiring.
It would be more-ideal to use cpu-time instead of wall clock time to avoid external resource contention for cpu or scheduling causing us to mistakenly trigger an execution timeout, but thread-specific cpu usage is not easily available on any OS (as far as I can tell...)
from logstash-filter-grok.
I think your last comment will have a very nice improvement for grok, specially when problematic situation happen.
+1 for them.
from logstash-filter-grok.
@suyograo can we close this as duplicate of #82 ?
from logstash-filter-grok.
Perhaps related - I have no idea why a simple regex times out - #99
from logstash-filter-grok.
this was implemented in #88
from logstash-filter-grok.
Related Issues (20)
- Allow grok pattern to be passed in as a parameter HOT 3
- Memory leak on 5.x
- Ability to auto-sort match options by frequency of match HOT 1
- getting values as an array HOT 1
- remove_field not working HOT 2
- "Prefix" functionality for grok HOT 4
- Track number of failed matches when using multiple pattern per field
- Warn when patterns don't have anchors HOT 1
- Significant Performance Regression using Jruby Timeouts HOT 8
- Implement ECS-Compatibility Mode HOT 4
- Logstash pipeline to remove passwords from log data HOT 1
- Using event fileds in configuration options HOT 1
- pure ruby regexp <capture:int> coercion does not work
- captures with same name won't coerce properly
- Logstash for iis SMTP
- Incoherent behavior of field references with overwrite
- [Test Failure] Syslog grok tests are failing on Logstash 8.x
- Behaviour when pattern writes to same input field (without "overwrite" option) HOT 1
- Add property to limit backtracking
- Regression on grok is case of match failure HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logstash-filter-grok.