Maybe support date parsing? about logstash-filter-grok HOT 8 CLOSED

I do think it's too complex. Have one filter do one thing at a time, generally speaking, and have it do it well.

Just because a thing can be done does not imply that it should, or must be done.

Simplicity often means saying, "No," to a lot of things.

from logstash-filter-grok.

I was noodling on the possible benefits. Doing this would let you embed the time format within your pattern, so SYSLOGBASE could have a defined timestamp pattern within it, and you could match:

  match => { "message" => "%{SYSLOGBASE} whatever" } 

and you get @timestamp set implicitly if SYSLOGBASE has a pattern within it that uses the proposed time parsing.

from logstash-filter-grok.

This would simplify tutorials ("%{COMBINEDAPACHELOG}" grok only, no date filter) etc

from logstash-filter-grok.

On the other hand, I like one filter doing one thing each.

But, the contrary, grok already parses text and does int/float conversion, why not time?

from logstash-filter-grok.

That idea does have merit, for pre-defined time formats. And if you can define those, the language to make it possible for others also makes sense.

Will this functionality depend on the presence of the date filter as well, then? If so, will it be JRuby-only, as the date filter depends on Joda?

from logstash-filter-grok.

The pipelining of filters makes a lot of sense to me. Trying to make one stage do too much can bog things down, and also make it harder to pin down "which filter is the bottleneck."

from logstash-filter-grok.

A counterpoint to "why not time?" I can think of is that it's unclear when the road ends there. "why not geoip?" etc.

I think my intent from this discussion is to find the reasons why time parsing would be nice, and try to generalize that and provide a good solution that, my gut says, won't involve grok feature changes. We have, for example, needs to provide a kind of 'parser pack' for known kinsd of logs (apache == grok + date + geoip + useragent) that would be useful to have solved.

from logstash-filter-grok.

This discussion was useful, but I Don't think we've decided what to do, and we haven't discussed it in a while. Will close.

We can always revisit this topic later :)

from logstash-filter-grok.