Logstash significantly slows down with longer grok pattern about logstash-filter-grok HOT 2 CLOSED

The short answer is that regular expressions and capturing (grok) is not cheap. It's possible that the slowdown is expected, but maybe not.

I see a lot of DATA uses in the parent ticket, and these can cause bad performance if used repeatedly throughout the pattern. The best way to achieve fast execution with grok is to as specific as possible with your patterns. DATA means .* which matches "anything or nothing" and is the least specific possible. Recommend approaching your pattern with tighter specifics to see what can be done.

Since this is old/idle. I'm going to close it. Any future concerns I am happy to help address in a future ticket :)

from logstash-filter-grok.

As extended to this issue, I need some help to parse similar logs in best efficient way:
I am using 1.4.2 logstash
Log1:

• "-" 2ebb4b4d.51e4940e11a3d [27/Aug/2015:16:57:33 +0530] "POST /request/url HTTP/1.1" 200 399 "-" "Zend_Http_Client" "-; ssl=-" 1 1307568 "9.9.9.9" "differentdata"

Log2:

• "-" 2ebb4b4d.51e4940e11a3d [27/Aug/2015:16:57:33 +0530] "POST /request/url HTTP/1.1" 200 399 "-" "Zend_Http_Client" "-; ssl=-" 1 1307568 "9.9.9.9"

My parse grok filter is

    grok {

            break_on_match => false
            patterns_dir => [ "/usr/local/infra/logstash/patterns" ]
            match => [


                   "message" , "%{APACHE_ACCESS_LOG} %{QUOTEDSTRING:apache-cookie} %{NUMBER:seconds_delete} %{NUMBER:response-time-in-microsec}  \"(?:%{IPORHOST:client-ip}|-)\"",

                   "message" , "%{APACHE_ACCESS_LOG} %{QUOTEDSTRING:apache-cookie} %{NUMBER:seconds_delete} %{NUMBER:response-time-in-microsec}  \"(?:%{IPORHOST:client-ip}|-)\" %{QS:differentdata}"


            ]
            remove_field => ["message","host","seconds_delete"]

}

I am sending output to elasticsearch and i am not seeing the "differentdata" field in elasticsearch. So there is no effect of the second pattern in grok. If I interchange the position of grok filter, Log1 output is not shown in elasticsearch.
So please help me with this.

from logstash-filter-grok.