Giter VIP home page Giter VIP logo

azure-health-data-services-toolkit's Introduction

Azure Health Data Services Toolkit

The Azure Health Data Services Toolkit helps you extend the functionality of Azure Health Data Services by providing a consistent toolset to build custom operations to modify the core service behavior. With the growth of health data workloads on Azure, we’ve found that developers need custom behavior on top of our services. This toolkit abstracts common patterns so you can focus on delivering your use cases.

NuGet Packages

Package Name Description
Microsoft.AzureHealth.DataServices.Core
NuGet
.NET 6 toolkit for creating custom operations when using Azure Health Data Services.
Microsoft.AzureHealth.DataServices.Channels.Extensions
NuGet
.NET 6 toolkit for extending channels using Azure Health Data Services.
Microsoft.AzureHealth.DataServices.Caching
NuGet
.NET 6 toolkit for adding caching using Azure Health Data Services.
Microsoft.AzureHealth.DataServices.Storage
NuGet
.NET 6 toolkit to simplify Azure storage operations when using Azure Health Data Services.

Getting started

The fastest way to test out the toolkit and see it in action is through our Quickstart sample. This sample will walk you through some common patterns that you'll need to create custom operations for Azure Health Data Services.

Read the developer guide for help setting up your local and cloud environment for developing custom behaviors for Azure Health Data Services.

Also check out our full list of samples on how to use the toolkit here for even more inspiration on how to create your own custom operations.

Common Fast Healthcare Interoperability Resources (FHIR®) use cases

Some FHIR service use cases that this toolkit can help you implement are:

  • FHIR operations not supported by the FHIR Service yet.
    • Trial implementation guides.
    • Organization-specific operations.
    • Less widely adopted operations.
  • Implementation guide development.
  • Transforming request and/or response payloads.
  • Custom authorization logic (like consent, etc.).

Key Concepts

For detailed information, read the concept guide here.

When we say “custom operations” we are talking about a purpose-built solution which acts as a proxy for a single or small set of HTTP endpoints. This toolkit is here to simplify developing such solutions. It’s recommended to use Azure API Management or similar for routing certain requests to these custom operations so that the client only sees one endpoint. Azure API Management can also present a unified authorization experience to your clients. This is why our samples don’t have authorization on the endpoints.

When building custom operations, you’ll come across these concepts related to the toolkit.

  • Operation Context: Common object passed between components of a pipeline containing the request and response.
  • Pipeline: Container for the actions of custom operations with filters, channels, and bindings executed in the order shown below.
    • Filter: A unit of action that modifies the request and/or result via the Operation Context. Filters can be chained together in a single input/output section of a pipeline.
    • Channel: Used to output data in a pipeline to an external system actor (ESA). This is usually an Azure service (like Storage, Event Hub, and/or Service Bus).
    • Binding: The target service for a custom operation (usually a FHIR service). This can be null for custom operations that don't need to have a destination.

What about the FHIR Proxy?

FHIR Proxy was created in response to customer requests for customizing the Azure API for FHIR. With the release of Azure Health Data Services, we’ve come up with a new approach to customization.

  • This toolkit lets you go beyond the proxy pattern and gives you tools for more extensive customization with programmatic components that flexibly connect to the broader Azure ecosystem.
  • This toolkit is designed to be used in smaller operation-specific modules. If you are customizing a certain behavior, you don’t need to proxy the rest of your API calls.
  • This toolkit is compute-agnostic and can be deployed on any .NET 6.0 server like Azure Functions, Azure App Service, Azure Kubernetes Service, etc.
  • This toolkit is released and versioned via NuGet packages.
  • We have designed this toolkit with coding best practices in mind, like object-oriented pipelines and extended testing.

If there is functionality in the FHIR Proxy that is not covered by the Health Data Services toolkit, please submit an issue and we will look into adding a sample!

Resources

Links

Sample production architecture

This architecture is a sample of how you could deploy and integrate custom operations built with the Azure Health Data Services toolkit in a production environment with Azure Health Data Services.

Example architecture diagram

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Disclaimers

The Azure Health Data Services toolkit is an open-source project. It is not a managed service, and it is not part of Microsoft Azure Health Data Services. You bear sole responsibility for compliance with local law and for any data you use with this open-source toolkit. Please review the information and licensing terms on this GitHub website before using the Azure Health Data Services toolkit.

The Azure Health Data Services toolkit GitHub is intended only for use in transferring and formatting data. It is not intended for use as a medical device or to perform any analysis or any medical function and the performance of the software for such purposes has not been established. You bear sole responsibility for any use of this software, including incorporation into any product intended for a medical purpose.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

FHIR® is the registered trademark of HL7 and is used with the permission of HL7.

azure-health-data-services-toolkit's People

Contributors

aliakseit avatar evachen96 avatar joalmeid avatar mikaelweave avatar mjschanne avatar pjirsa avatar sordahl-ga avatar v-pbartley avatar v-vijadhav avatar vmsuthar56 avatar w-matt-long avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-health-data-services-toolkit's Issues

RestBinding security token is always null

The RestBinding security token will always be null, i.e., lines 88-92.

The code should be more below such that the security token will be set if the IAuthenticator is not null.

string securityToken = null;
if (authenticator != null)
{
string userAssertion = authenticator.RequiresOnBehalfOf ? context.Request.Headers.Authorization.Parameter.TrimStart("Bearer ".ToCharArray()) : null;
securityToken = await authenticator.AcquireTokenForClientAsync(options.Value.ServerUrl, options.Value.Scopes, null, null, userAssertion);
}

Content-Type header cannot contain charset

Describe the bug
When a request is provided which contains a charset within the Content-Type header — e.g. application/fhir+json; charset=utf-8 — an exception is thrown due to the following line:

message.Content.Headers.ContentType = new MediaTypeHeaderValue(header.Value.ToArray()[0]);

Expected behavior
Requests are handled without issue.

Actual behavior
The following exception is thrown:

---> System.FormatException: The format of value 'application/fhir+json; charset=utf-8' is invalid.
  at System.Net.Http.Headers.MediaTypeHeaderValue.CheckMediaTypeFormat(String mediaType, String parameterName)
  at System.Net.Http.Headers.MediaTypeHeaderValue..ctor(String mediaType)
  at Microsoft.AzureHealth.DataServices.Pipelines.AzureFunctionExtensions.ConvertToHttpRequestMesssage(HttpRequestData req)
  at Microsoft.AzureHealth.DataServices.Pipelines.AzureFunctionPipeline.ExecuteAsync(HttpRequestData request)
  at Bupa.CC.Fhir.Consent.Function.FhirFunction.ConnectedCareFhirHandler(HttpRequestData req, String fhirRoute, FunctionContext executionContext) in /src/src/Bupa.CC.Fhir.Consent.Function/FhirFunction.cs:line 32
  --- End of inner exception stack trace ---

Environment summary
SDK Version: 1.0.1
OS Version: Linux
Compute Platform: Azure Function

Additional context
Judging by the code, I believe the same issue still exists within the 1.1.0 version of the library. Given that the version bump introduces breaking changes and will require additional efforts to utilise within our project, I believe hotfixes should be published for both current versions.

Quickstart function - PatientPost returns 502 Bad Gateway

After deploying latest changes to Quickstart Function, Azure Function App becomes unstable. Portal UI is only able to detect deployed functions intermittently. Calling PatientPost function with sample Patient.json returns 502 Bad Gateway errors with trace logs reporting - [Error] Failed to start a new language worker for runtime: dotnet-isolated.

Expose Response Headers to Pipeline

Is your feature request related to a problem? Please describe.
For using the Toolkit with Azure AD, we need the ability to modify the response headers. This is necessary for modifying the Azure AD flow and having clients redirect to the newly formed AAD URL with a Http redirect.

Describe the solution you'd like
It would be great if the OperationContext for the pipeline exposes a headers collection. This could be initialized as empty, binding could conditionally update this (by binding config), then output filters could modify the header collection.

Describe alternatives you've considered
Shim on top of the pipeline inside my program. Works but doesn't keep code segmented in the filter.

Need a way to not execute the built in REST Binding

Is your feature request related to a problem? Please describe.
Currently, there is no way to stop the execution of the REST binding if an input filter signals a Failure or already has the status code set (e.g. for a redirect).

Describe the solution you'd like
Update to the rest binding to check if the pipeline has failed or if status code already set. This could be done like the filters where they have a "run type".

RestBinding to non-existing url should not return 500 but 404

Describe the bug
When creating a custom operation for a single record (like Patient/), the RestBinding will throw a 500 if a client requests a record that does not exist.

To Reproduce

  1. Create a web or function pipeline with a binding that points to a FHIR server (+ an authenticator)
  2. Send a request to the FHIR server through the pipeline for a record that doesn't exist (like Patient/123).

Expected behavior
There should at least be an option to not fail on HTTP status codes between 202 and 499 but to pass this through. Especially for 300-400. I don't think "EnsureStatusSuccessful in the RestBinding is correct

Actual behavior
MicrosoftTeams-image

Comment Error samples/FeatureSamples/ServiceBusChannel/Program.cs

An error/nit in the comment in samples/FeatureSamples/ServiceBusChannel/Program.cs
the first line of the comment is "Adds Service Busas", which should be "Add Service Bus"
the second line of the comment is "too large for Storage Bus.", which should "too large for the Service Bus SKU."

   // Adds Service Busas the first input channel. Azure Storage is a backing store for events that are
    // too large for Storage Bus.
    services.AddInputChannel<ServiceBusChannelOptions>(typeof(ServiceBusChannel), options =>
    {

WebApi calls FHIR Service with User Credentials

Tell us what business problem you are trying to solve
We are developing a WebApi that consumes a FHIR service, running the application using 'DefaultAzureCredential()' with a user logged in through Azure CLI works fine.

builder.Services.AddBinding<RestBinding, RestBindingOptions>(options =>
{
  options.BaseAddress = new Uri(configuration.GetValue<string>("FhirServerUrl"));
  options.Credential = new DefaultAzureCredential();
});

Next step is protecting the WebApi controllers using [Authorize] attribute, we receive the bearer token in the API that grants the access to it and the operations the user can perform. The scenario is like that:

User -> Api -> FHIR Server

How do we manage the credentials to call FHIR server in the name of the user?

I guess the API should request for a new token in the name of the user, and somehow use it in the WebPipeline.

Describe what behavior a customization to Azure Health Data Services would solve this problem*
Would be nice to have a sample of this situation, I think it is pretty common.

Does this issue belong as a feature to Azure Health Data Services instead?
No

Additional context
None

Thanks!

Vague Exception When Using Identity Custom Header without Authorization Header

Describe the bug

When using an identity type custom header without an Authorization header, the returned exception is vague and hard to debug.

'System.NullReferenceException: 'Object reference not set to an instance of an object.''

To Reproduce
Steps to reproduce the behavior. If you can include code snippets or links to repositories containing a repro of the issue that can helps us in detecting the scenario it would speed up the resolution.

  1. Create a pipeline with headers
  2. Add an Identity Custom Header to the pipeline config
  3. Feed a HttpRequest without the Authorization header

Expected behavior
There should be an exception thrown that clearly states the Authorization header is missing.

Actual behavior
Exception is thrown without much of a stack and a vague message.

Environment summary
SDK Version: Current
OS Version: Windows 10
Compute Platform: Local ASP.NET

Additional context

Stack Trace not showing details:

 at Azure.Health.DataServices.Clients.Headers.HttpCustomHeaderCollection.AppendAndReplace(HttpRequestMessage request)\r\n   at HeaderInputFilter.<ExecuteAsync>d__12.MoveNext() in Z:\\source\\health-data-services-sdk\\samples\\FeatureSamples\\CustomHeaders\\HeaderInputFilter.cs:line 27

Add Token Passthrough to RestBinding

Is your feature request related to a problem? Please describe.
Currently working on a SMART on FHIR implementation using the toolkit. For simplicity of use, it would be best for us to pass through the access token from the client to the FHIR Service instead of additional AAD setup for the on-behalf-of flow (which gives us nothing here).

Describe the solution you'd like
I would like the ability to pass through the access token from the request to the REST Binding.

Additional context
Authenticator, rest request builder, and rest binding will all need to be updated.

Filter Error Handlers Incorrect

The filter handler methods are incorrect - they have Channel in them. Also the body parameter of FilterErrorEventArg is ignored.

private void OutputFilter_OnFilterError(object sender, FilterErrorEventArgs e)
        {
            context.IsFatal = true;
            context.StatusCode = e.Code ?? HttpStatusCode.InternalServerError;
            context.Error = e.Error;
            logger?.LogError(e.Error, "Pipeline {Name}-{Id} output filter {ChannelName}-{ChannelId} error.", Name, Id, e.Name, e.Id);
            OnError?.Invoke(this, new PipelineErrorEventArgs(Id, Name, e.Error));
        }

        private void InputFilter_OnFilterError(object sender, FilterErrorEventArgs e)
        {
            context.IsFatal = true;
            context.StatusCode = e.Code ?? HttpStatusCode.InternalServerError;
            context.Error = e.Error;
            logger?.LogError(e.Error, "Pipeline {Name}-{Id} input filter {ChannelName}-{ChannelId} error.", Name, Id, e.Name, e.Id);
            OnError?.Invoke(this, new PipelineErrorEventArgs(Id, Name, e.Error));
        }

Replace Newtonsoft with System.Text.Json for Performance

While talking to @sordahl-ga, he highlighted that Newtonsoft is slower in comparison to System.Text.Json. We switch over for the performance gains and align with the Microsoft recommendation of using System.Text.Json.

We may need to change Microsoft.AzureHealth.DataServices.Json or delete it altogether to accomplish this.

Inconsistent Method Handling for FHIR Uri Functions

FhirUriPath takes the HTTP method as a string while UpdateFhirRequestUri takes a value from the HttpMethod Enum. Both should be using the HttpMethod Enum.

RestRequestBuilder also takes a string and should take the Enum.

Quickstart Readme

In the Deploy Resources section of the Quickstart Readme, the line "Deploy the needed resources by running: azd provision" should be preceded by an line telling the user to run "az login"; otherwise if the user is not logged in then the "azd provision" command will fail.

Deploy Resources
Clone this repository and open a terminal or command window to this folder samples/Quickstart.
Deploy the needed resources by running: azd provision
This will take about 20 minutes to deploy the FHIR Service.
If you have run this sample in the past, using the same environment name and location will reuse your previous resources.

Sample Quickstart Function 415 Unsupported Media Type

Describe the bug
When calling the POST /Patient endpoint, the Azure Function is returning the following error regardless of Content-Type specified in the header.

Actual behavior

HTTP/1.1 415 Unsupported Media Type
cache-control: private
content-security-policy: frame-src 'self';
content-type: application/fhir+json; charset=utf-8
date: Thu, 31 Aug 2023 19:33:40 GMT
request-context: appId=
strict-transport-security: max-age=31536000; includeSubDomains
transfer-encoding: chunked
vary: Origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-request-id: 9611f686bf3abb4e8f50b18ade89398c
    
{
    "resourceType": "OperationOutcome",
    "id": "9611f686bf3abb4e8f50b18ade89398c",
    "meta": {
        "lastUpdated": "2023-08-31T19:33:39.9074508+00:00"
    },
    "issue": [{
        "severity": "error",
        "code": "not-supported",
        "diagnostics": "Value supplied for the \"Content-Type\" header is not supported."
    }]
}

Quickstart Sample Readme

Prereq section specifies "azd --version", which should be "azd version"

Prerequisite check
In a terminal or command window, run dotnet --version to check that the .NET SDK is version 6.0 or later.
Run az --version and azd --version to check that you have the appropriate Azure command-line tools installed.

Quickstart template repo no longer exists

Describe the bug
When following the getting started section, running azd init --template Azure-Samples/azure-health-data-services-toolkit-fhir-function-quickstart fails because that repo no longer exists.

To Reproduce
see description

Expected behavior
azd init command should complete successfully and create my new local project.

Actual behavior
fails with:

ERROR: init from template repository: fetching template: failed to clone...
remote: Repository not found.

Poor Performance using Authenticator and RestBinding

The built in RestBinding asks for a new access token for every request. This is not performant and is a blocker to any production scale workload with this toolkit.

Moreover, the RestBinding creates a new HttpClient for every request which has some performance concerns.

For better performance of the client, I would recommend:

  • Using a singleton HttpClient vs creating one for every request. (see here).

For better performance for token use, I could recommend code to cache an access token and fetch a new one on failure. Azure KeyVault SDK does a good job at this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.