azure / azure-functions-dotnet-worker Goto Github PK

I ran into an issue while testing -- we were throwing a NullRef in here: https://github.com/Azure/azure-functions-dotnet-worker/blob/main/src/DotNetWorker/Definition/DefaultFunctionDefinitionFactory.cs#L20, but I wasn't seeing any log in the host. We need to ensure that exceptions in code like this make it back to the host for logging/debugging.

It seems that any error today with source generation still makes the build be successful and leaves a small info message saying that source wasn't generated. We should force the build to fail in such cases with some helpful message.

We can catch all exceptions and use SourceGeneratorContext.ReportDiagnostic to create a diagnostic with a severity of error.

See example here -- we should use ObjectSerializer from the Azure SDK to abstract away serialization technology. This likely should happen for GA because it could impact public API and slightly change behavior.

See here for an example: https://github.com/Azure/azure-sdk-for-net/blob/ef4430baed/sdk/core/Microsoft.Azure.Core.NewtonsoftJson/README.md. We'd likely add a Serializer property to our Options.

We need to to ensure the dispatch path is not just a "fire and forget". Messages need to be tracked.

When any binding is unavailable such that the datatype that is received is null, we get a NotSupportedException. We need to also handle the case for TypedData.DataOneofCase.None.

We probably need to handle a bunch of other types as well in future. But None, I think, is more crucial than others.

Today we use Source Generators... but this won't work for non-C# or if the assembly has been AOT compiled. Need to come up with alternative or fallback...

In order to support constructor injection, we need to activate the function class using the container to fulfill constructor parameters.

Issue to track the following:

• Review existing detectors to identify whether any of them would need to be updated
• Identify new detectors needed

All ARM calls related to "/function(s)" API are missing binding information for .NET 5 worker. These API uses the payload created by the Functions Host during a sync triggers call.

When a sync triggers call is made, Functions Host somewhat relies on the physical function.json files to create the config portion of the request. This causes binding information to be missing in Functions that come outside of function.json. Here's the code that relies on function.json.

We need to decouple the call from the physical file and use the FunctionMetadata. All this work needs to happen in the Functions Host, but tracking here because they are crucial for .NET 5 support.

• Add templates
• Configure local.settings.json to start worker
• func azure functionapp publish to configure worker (build the worker first)

Preface: As the title notes, this is probably more of a discussion than an issue, but it doesn't look like discussions are enabled on this repo yet... which is understandable since it's only just recently (i.e. 3 days ago) gone into public beta. 😊

In looking at the samples and what immediately jumped out at me as someone who utilizes HTTP triggered functions the majority of the time is the introduction of two new data contract classes to represent the request and response: HttpRequestData and HttpResponseData.

First, I would like to ask that we could have some kind of short explanation of why these classes were introduced. I believe it's because the SDK does not want to pin itself to a specific version of System.Net.Http, is that correct? If so, would you care to expand on why you think it's a bad idea to tie a specific extension to another dependency? Frankly, I think it would be a big mistake to force new types on .NET developers in the context of an Azure Function when in so many other contexts System.Net.Http is what we all use and which itself has an extension ecosystem built around it.

Second, if it comes down to decoupling, well then I have concerns about the design of these contracts. For example, the Body being a string: this is dangerous for performance on multiple levels. First and most basic, even if I actually do want to return a string... I shouldn't have to build a whole contiguous string like that. Second, if I don't want to return a string and want to return binary data... what option do I have now? I would have to return base-64 encoded bodies and that's just, well, not a good idea from an HTTP perspective and could completely box out some standard HTTP scenarios like file downloads or serving images. I would also suggest that the Headers property being defined as the concrete type Dictionary<string, string> and always allocated even if not used is a terrible idea from a performance perspective. Anyway, we can split off and go into deeper on into this in separate issues once we have the core discussion established, but the point is that so much of this is already well thought out, designed and implemented into the existing System.Net.Http type system and I think it's a mistake to eschew that for an extension where I'm very much opting in to using HTTP in the first place.

In the functions host, everything uses ILogger, which makes it hard to differentiate between System and User logs. We should take a different approach in the worker and make it explicit that logs written by system components do not use the "vanilla" ILogger, but maybe some internal wrapper class that allows us to differentiate.

We're going to try to skip over the need for function.json files in dotnet worker projects. In order to do that, we are hoping that we can auto-generate a class that implements IFunctionProvider, which will provide the host with all the function metadata it needs without having an actual json file artifact as part of the deployment.

There are open compliance tasks that need to be reviewed for your azure-functions-dotnet-worker repo.

Action required: 4 compliance tasks

To bring this repository to the standard required for 2021, we require administrators of this and all Microsoft GitHub repositories to complete a small set of tasks within the next 60 days. This is critical work to ensure the compliance and security of your Azure GitHub organization.

Please take a few minutes to complete the tasks at: https://repos.opensource.microsoft.com/orgs/Azure/repos/azure-functions-dotnet-worker/compliance

• The GitHub AE (GitHub inside Microsoft) migration survey has not been completed for this private repository
• No Service Tree mapping has been set for this repo. If this team does not use Service Tree, they can also opt-out of providing Service Tree data in the Compliance tab.
• No repository maintainers are set. The Open Source Maintainers are the decision-makers and actionable owners of the repository, irrespective of administrator permission grants on GitHub.
• Classification of the repository as production/non-production is missing in the Compliance tab.

You can close this work item once you have completed the compliance tasks, or it will automatically close within a day of taking action.

If you no longer need this repository, it might be quickest to delete the repo, too.

GitHub inside Microsoft program information

More information about GitHub inside Microsoft and the new GitHub AE product can be found at https://aka.ms/gim or by contacting [email protected]

FYI: current admins at Microsoft include @jeffhollan, @pragnagopa, @mhoeger, @fabiocav, @shimedh, @ahmelsayed, @mathewc, @cgillum, @brettsam, @anirudhgarg, @paulbatum, @davidmrdavid, @ehamai

Currently the preview of Microsoft.Azure.Functions.Worker only targets net5.0.
Is there a plan to support netcoreapp3.1 or netstandard2.0/2.1?

We'd like to be able to update our functions to this new out-of-process mode, but we won't be able to use .NET 5 as it is not an LTS version (EOL is too soon). It would be great if we wouldn't have to wait until .NET 6 is released.

The DefaultFunctionDefinitionFactory is a type that should be generated at build time

Right now we're auto-wiring up the invoker as the last piece of middleware. We should make this explicit and expose a UseFunctionExecutionMiddleware() that we wire-up in Program.cs.

Please follow the instructions in the OneNote to onboard your language worker.

@fabiocav -- should we be signing these assemblies? I think we can just use our own key now, right?

Make sure that it doesn't expose anything GRPC-related and that it is extensible for future usage.

We need something like dotnetworker to differentiate from dotnet -- but needs to support future versions of .NET

• Plan update to stacks that allows us to specify the language, version, and "mode" (embedded or isolated) for .NET
• Talk with Elliot/Allison about whether we can update the UX to look more like Web Apps:

Functions:

Web Apps:

• This would remove the need for a third dropdown in the Function create flow and move everything to one selection. For example, we'd have ".NET Core 3.1 embedded", ".NET Core 3.1 isolated", and ".NET 5 isolated"

Some limitations to note:

• There may be breaking changes in the future
• Only core tools workflow supported
• Rich bindings not supported (give examples) - durable, etc

If host.json has extension bundle configured, everything works fine except no functions show up. It's really hard to tell the root cause. And this can happen if your app is missing host.json as even the default one has extension bundles.

We likely need to fix this in Host, but tracking here for .NET worker work.

Currently, Functions worker is configured with UseFunctionExecutionMiddleware() to provide the necessary infrastructure for triggers and output bindings. I'd like to be able to test my functions w/o running the full middleware when using the container DI based Functions. This will allow simpler testing scenarios that do not require Functions runtime to be installed and bootstrapped.

Rich types like CloudBlockBlob, etc, don't work when used in the worker. We need to define how to handle this so it's not a runtime exception.

Getting this error:

The 'XXX' function is in error: Can't figure out which ctor to call.

Today we're just assuming that functions are synchronous and we're not awaiting the call.

This includes creating the function class with the DI services (if not static).

Minimize surface area for v1. Make sure that anything referencing GRPC or that couples us to System.Text.Json isn't exposed. Things like Converters can be all internal and exposed later to give us more time for API design.

After building an app, we end up with 2 runtimes folders in the output:

• bin/Debug/net5.0/runtimes
• bin/Debug/net5.0/bin/runtimes

The second one only contains win and win7-x64 folders. Needs Linux and OSX ones too.

right now if you use "dotnet5" (or anything unknown) -- the core tools will overwrite the app setting when publishing

FunctionName with hyphen (-) breaks build

To repro, take a working example and just change the function name to something like this:

[FunctionName("CosmosData-Get")]

This is the results that show up in the failed build:

11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(42,59,42,60): error CS1002: ; expected
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(42,49,42,59): error CS0818: Implicitly-typed variables must be initialized
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(42,60,42,63): error CS0103: The name 'Get' does not exist in the current context
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(49,12,49,15): error CS0103: The name 'Get' does not exist in the current context
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(56,66,56,69): error CS0103: The name 'Get' does not exist in the current context
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(56,55,56,111): error CS0201: Only assignment, call, increment, decrement, await, and new object expressions can be used as a statement
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(59,57,59,60): error CS0103: The name 'Get' does not exist in the current context
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(59,46,59,102): error CS0201: Only assignment, call, increment, decrement, await, and new object expressions can be used as a statement
11>C:\Dev\ServiceExtensions\src\Example.Function.Net5\Microsoft.Azure.Functions.Worker.Sdk\SourceGenerator.FunctionProvider\_GeneratedFunctionProvider.cs(60,29,60,32): error CS0103: The name 'Get' does not exist in the current context
11>Done building project "Example.Function.Net5.csproj" -- FAILED.

Simply removing the hyphen in the name will allow this to build properly

We need to bring back some of the abstractions we previously had in the worker POC.

Before, parts of a url could be bound to, which could then be used in other bindings.

E.g. you could take a part of the HTTP trigger url, and use that for a binding to a storage item.

Or see this sample.

Will this still be possible?

Today we explicitly pass an ILogger to the ExecutionContext.

• We need to wire this up as an ILoggerProvider, which allows ILoggers used in customer DI services to log back to the host.
• Investigate how we handle categories, etc, from these loggers as they pass back through the host. For example, does the host respect the category on the RpcLog type and use it for its own logs?