Comments (8)
I do think it's too complex. Have one filter do one thing at a time, generally speaking, and have it do it well.
Just because a thing can be done does not imply that it should, or must be done.
Simplicity often means saying, "No," to a lot of things.
from logstash-filter-grok.
I was noodling on the possible benefits. Doing this would let you embed the time format within your pattern, so SYSLOGBASE could have a defined timestamp pattern within it, and you could match:
match => { "message" => "%{SYSLOGBASE} whatever" }
and you get @timestamp
set implicitly if SYSLOGBASE has a pattern within it that uses the proposed time parsing.
from logstash-filter-grok.
This would simplify tutorials ("%{COMBINEDAPACHELOG}" grok only, no date filter) etc
from logstash-filter-grok.
On the other hand, I like one filter doing one thing each.
But, the contrary, grok already parses text and does int/float conversion, why not time?
from logstash-filter-grok.
That idea does have merit, for pre-defined time formats. And if you can define those, the language to make it possible for others also makes sense.
Will this functionality depend on the presence of the date filter as well, then? If so, will it be JRuby-only, as the date filter depends on Joda?
from logstash-filter-grok.
The pipelining of filters makes a lot of sense to me. Trying to make one stage do too much can bog things down, and also make it harder to pin down "which filter is the bottleneck."
from logstash-filter-grok.
A counterpoint to "why not time?" I can think of is that it's unclear when the road ends there. "why not geoip?" etc.
I think my intent from this discussion is to find the reasons why time parsing would be nice, and try to generalize that and provide a good solution that, my gut says, won't involve grok feature changes. We have, for example, needs to provide a kind of 'parser pack' for known kinsd of logs (apache == grok + date + geoip + useragent) that would be useful to have solved.
from logstash-filter-grok.
This discussion was useful, but I Don't think we've decided what to do, and we haven't discussed it in a while. Will close.
We can always revisit this topic later :)
from logstash-filter-grok.
Related Issues (20)
- Memory leak on 5.x
- Ability to auto-sort match options by frequency of match HOT 1
- getting values as an array HOT 1
- remove_field not working HOT 2
- "Prefix" functionality for grok HOT 4
- Track number of failed matches when using multiple pattern per field
- Warn when patterns don't have anchors HOT 1
- Significant Performance Regression using Jruby Timeouts HOT 8
- Implement ECS-Compatibility Mode HOT 4
- Logstash pipeline to remove passwords from log data HOT 1
- Using event fileds in configuration options HOT 1
- pure ruby regexp <capture:int> coercion does not work
- captures with same name won't coerce properly
- Logstash for iis SMTP
- Incoherent behavior of field references with overwrite
- [Test Failure] Syslog grok tests are failing on Logstash 8.x
- Behaviour when pattern writes to same input field (without "overwrite" option) HOT 1
- Add property to limit backtracking
- Regression on grok is case of match failure HOT 4
- Pipeline crashes with undefined method `each' for nil:NilClass error in event filter method
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logstash-filter-grok.