User-Agent parsing in redir info about redelk HOT 10 CLOSED

Agree, we should move this. Putting this for milestone v2.0.0-beta.4 (when we also revisit the exporting of templates so this can be done in one sweep)

from redelk.

User-Agent parsed data is for now put in [source][host_info]. To adhere to ECS, the different fields should be moved to [source][host][os] corresponding fields. To be done when maindev branch is stable enough.

from redelk.

Not sure Im following you, should all useragent info be put under source.host? Meaning the items in the image below should be renamed from source.host_info to source.host?

I cant see any proper reference by ECS except that this info should be stored in the root as user_agent.

from redelk.

This is the information for user_agent from ECS: https://www.elastic.co/guide/en/ecs/current/ecs-user_agent.html
I've checked but they don't say it if should be on root or under another key.
The best would be to put it under source I think.

We would end up with something like source.user_agent.os.name, etc (os being a sub-key under user_agent: https://www.elastic.co/guide/en/ecs/current/ecs-os.html)

Does that make sense?

from redelk.

So the only renaming required would be source.host_info -> source.user_agent? I can live with that.

Renaming is required in the Logstash config files as well as in the index templates of ES and Kibana.

from redelk.

Yes, and changing user_agent.os to user_agent.os.name

Not sure if user_agent.os and user_agent.os.name are always the same (from your screenshot)?

from redelk.

That naming of os and os_name is part of the Logstash user-agent module. The difficulty here is that the ECS user-agent documentation is not aligned with the fiels that the plugin actually adds.

Not much we can steer here. The only thing that can be done is setting the target (https://www.elastic.co/guide/en/logstash/current/plugins-filters-useragent.html#plugins-filters-useragent-target) Of course we could do post field renaming. But imho this beats the purpose of it. If it is not conform ECS Ill leave it to Elastic to update their own Logstash plugin to adhere to ECS :-)

from redelk.

This should be in 1 PR preferably with the change of the corresponding alarms!
@xychix can colaborate to make this work in 1 PR.

from redelk.

@xychix not sure about that.

@fastlorenzo see my comment on Nov 17. Do you agree that it would be ok to leave it as it is right now, and only to revisit once Elastic fixes the Logstash user-agent module to adhere to ECS?

from redelk.

@MarcOverIP I think that should be fine indeed, it is not too confusing anyways and it doesn't look like Kibana is making use of that natively.

from redelk.