(This issue was originally filed by <a class="user-mention notranslate" data-hovercard

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Grok: Maybe allow defining the type in the pattern definition about logstash-filter-grok HOT 3 CLOSED

logstash-plugins commented on June 21, 2024

Grok: Maybe allow defining the type *in* the pattern definition

from logstash-filter-grok.

Comments (3)

avleen commented on June 21, 2024

We solved this a different way internally, by using notation on the Elasticsearch mapping.
We have the following dynamic field mapping:

"dynamic_templates": [
    {
        "short_template" : {
            "match" : "s_*",
            "mapping" : { "type" : "short", "index" : "not_analyzed" }
        }
    },
    {
        "long_template" : {
            "match" : "l_*",
            "mapping" : { "type" : "long", "index" : "not_analyzed" }
        }
    },
    {
        "double_template" : {
            "match" : "d_*",
            "mapping" : { "type" : "double", "index" : "not_analyzed" }
        }
    },
    {
        "bool_template" : {
            "match" : "b_*",
            "mapping" : { "type" : "boolean", "index" : "not_analyzed" }
        }
    },
    {
        "string_template" : {
            "match" : "*",
            "mapping": { "type": "string", "index": "not_analyzed" },
            "match_mapping_type" : "string"
        }
    }
],

If a field name starts with l_, Elasticsearch maps it to a long.
If it starts with b_, it's a boolean.
It requires more work on the end user to make it happen, and will require people to rename their fields.
But it's performant and reliable.

from logstash-filter-grok.