analysis sandbox inject_message: Can't set message Hostname about hindsight HOT 6 CLOSED

Analysis plugins should be generating new content (data transformations should happen in the inputs/outputs). If one is allowed to 'fixup' a message and make it look like it came from the original source this would be less than idea especially if untrusted code/users are allowed to perform dynamic analysis.

from hindsight.

Yes I could do it in the heka_tcp.lua input plugin, but I would prefer keep it clean.
If I use an analysis plugin, I can easily leverage the message_matcher config feature to filter the input messages that I want to decorate/modify rather than manually matching input messages directly in the heka_tcp.lua sandbox.
And if I have multiple input plugins which could create similar messages, I would have to copy the same code in every input plugins, whereas I could have done it once in the analysis plugin.

Also if it is a security consideration, when untrusted users are able to perform dynamic analysis, they are also able to load dynamic input/output plugins where they can rightfully "fixup" a message and mutate the hostname...

I would have thought that using analysis sandboxes for filtering or transforming flying messages was a sensible use case.

from hindsight.

• You can use modules, the closest example I have is the encoder module here: https://github.com/mozilla-services/lua_sandbox/blob/master/sandboxes/heka/output/elasticsearch_bulk_api.lua#L38
• No, public access to dynamic loading will be sitting behind a web UI and input/output plugins will have different permissions (if allowed at all) than analysis plugins.
• It is a reasonable use case and you can... you just cannot make it look like it came from some other host after changing it :). I am not an absolute 'no' on the issue. Since no message can actually be trusted this is probably a moot point. I will leave the issue open... other comments/opinions welcome.

from hindsight.

Ok, I understand your reasoning.
Since my use case is a bit different, would you accept a patch to make the Logger/Hostname override configurable in the global hindsight.cfg file (and defaulting to the current behavior)?

from hindsight.

Finally I've decided to use a custom field to pass the original Hostname, and to update my output plugins to use it instead of the message Hostname.
It works and this workaround is ok with me, so I'm closing this issue.

from hindsight.

the analysis sandbox will take a cfg option to allow this; use case: implementing a Heka Go decoder sandbox.
The default will be to overwrite the Headers provided or not (the current behaviour)
The override will accept whatever the user provides (including nil)

Edit: note output plugins actually set the restricted headers if they are nil so lets make it consistent

from hindsight.