Unable to overwrite log formatting into a console for Azure Function Isolated Host Worker about azure-functions-dotnet-worker HOT 15 OPEN

@kAleksei thank you, that is helpful to understand your scenario. I am familiar with k8s log pattern like this and sounds like a solid reason for us to evaluate supporting this. Unfortunately, it won't be possible now. But we have been discussing sweeping improvements for observability in Azure Functions and this can be considered as a part of it. There is no ETA on that and the work has not been started yet.

from azure-functions-dotnet-worker.

Hey, any comments on this?

from azure-functions-dotnet-worker.

We ended up filtering this out at promtail level, but that does not feel right.

from azure-functions-dotnet-worker.

It will not work for me, because I don't use application insights, but read logs from stdout.

from azure-functions-dotnet-worker.

To explain what is going on, standard out is piped to the host process and then it is written to the console. This is where that extra info: Host.Function.Console[0] comes from. No immediate plans to change that behavior, but we are looking into improving our logging abilities and may consider a change for this in the future.

from azure-functions-dotnet-worker.

Thanks for answering @jviau ! And just to clarify, we as developers have no control over this, and the only way to remove it is in our log scaping tool (promtail in my case)?

from azure-functions-dotnet-worker.

@jviau, it would be nice if this could be prioritized because we are writing the logs in JSON format single line. And every line is a separate log entry. Therefore every log that we want to write multiplies by two as functions host logs category and line breaks. As a result, logging costs are twice as high.

from azure-functions-dotnet-worker.

@dalibormesaric correct there is no control over this at the moment. The worker process is created as a child of the host process and as part of that the stdout is redirected to the host process.

@kAleksei we have no concrete plans to address this at the moment. Is using stdout your only option? Can you add a custom ILoggerProvider and ILogger which outputs to some other handle or stream?

from azure-functions-dotnet-worker.

@jviau unfortunately, this is only the option to log into stdout.

from azure-functions-dotnet-worker.

Can you help me understand your scenario a bit more? Why is it important to log to stdout? What system are you using which relies on stdout?

from azure-functions-dotnet-worker.

@jviau, we are deploying the azure functions isolated to k8s cluster. In the cluster, we have installed an agent, which is responsible for pulling logs from stdout of the container. So more than 1k different services are using this approach.
We are using datadog agent for gathering the logs. Here are the docs how it actually works: https://docs.datadoghq.com/containers/kubernetes/log/?tab=operator

from azure-functions-dotnet-worker.

@jviau, I appreciate your understanding. If you require any further information, please don't hesitate to ask. Additionally, I am eagerly awaiting updates on this item, as this feature holds a high level of importance for us.

from azure-functions-dotnet-worker.

@jviau I believe I have a problem related to the topic. In my case, I cannot remove the datetime stamp in front of the json, this ends up impacting elastic collection, which only accepts files in json format.

from azure-functions-dotnet-worker.

@domingosliraneto, did you have a chance to run your functions without IDE (like Visual Studio tools) and azure-functions-core-tools? For instance in docker container. Looks like this output formatted and written to console with debugging tools.

from azure-functions-dotnet-worker.

, você teve a chance de executar suas funções sem IDE (como ferramentas do Visual Studio) e ? Por exemplo, no contêiner docker. Parece que esta saída formatada e gravada no console com ferramentas de depuração.azure-functions-core-tools

Yes, I tested it in a container and the effect is the same as shown by @domingosliraneto

Remember, in the structured log configuration, dates are not defined. The same pattern is even used in other worker applications, for example.

from azure-functions-dotnet-worker.