Giter VIP home page Giter VIP logo

azure-functions-datalake-extension's Introduction

Azure Data Lake Storage Gen1 Binding for Azure Functions

The following binding can be used with Azure Functions v2 C# Class Library to connect to Azure Data Lake Storage Gen1.

Instructions

Clone repo and add a reference to the WebJobs.Extensions.DataLake project.

using Microsoft.Azure.WebJobs.Extensions.DataLake;

Output Binding

Add the following attributes that include the account FQDN, ApplicationId, Client Secret and Tenant Id.

[DataLakeStore(
  AccountFQDN = @"fqdn", 
  ApplicationId = @"applicationid", 
  ClientSecret = @"clientsecret", 
  TenantID = @"tentantid")]out DataLakeStoreOutput dataLakeStoreOutput

View a sample function using output binding.

Input Binding

Add FileName property to retrieve a specific file from your Datalake Store.

[DataLakeStore(
  AccountFQDN = @"fqdn", 
  ApplicationId = @"applicationid",
  ClientSecret = @"clientsecret",
  TenantID = @"tentantid",
  FileName = "/mydata/testfile.txt")]Stream myfile

View a sample function using input binding.

Binding Requirements

  1. Azure Data Lake Store Gen1
  2. Setup Service to Service Auth using Azure AD
  3. Azure Functions and Webjobs tools extension
  4. Add the application settings noted below.

local.settings.json expected content

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "AzureWebJobsDashboard": "UseDevelopmentStorage=true",
    "fqdn": "<FQDN for your Azure Lake Data Store>",
    "tentantid": "<Azure Active Directory Tentant for Authentication>",
    "clientsecret": "<Azure Active Directory Client Secret>",
    "applicationid": "<Azure Active Directory Application ID>",
    "blobconn": "<Azure Blobg Storage Connection String for testing Blob Trigger>"
  }
}

End to End Testing

If you wish to run and or make modifications to the E2E testing you will need to create an appsettings.json with all the required settings. Use the standard format for values instead of the functions formatting.

E2E appsettings.json expected content

{
  "AzureWebJobsStorage": "UseDevelopmentStorage=true",
  "AzureWebJobsDashboard": "UseDevelopmentStorage=true",
  "fqdn": "<FQDN>",
  "tentantid": "<Tentant ID>",
  "clientsecret": "<Client Secret>",
  "applicationid": "<Application ID>",
  "blobconn": "<Blob Storage Connection string for Trigger>"
}

License

This project is under the benevolent umbrella of the .NET Foundation and is licensed under the MIT License

Contributing

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-functions-datalake-extension's People

Contributors

joescars avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar msftgits avatar paulbatum avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-functions-datalake-extension's Issues

Nuget package

I cannot find this on nuget if I'm correct. Is it possible to release a nuget package so we can easily use this as an output for our functions app?

Data Lake gen2 binding

Is it possible to use ADLS gen 2 binding (dfs.core.windows.net)? I get this error:
Content-Type of error response: application/json; charset=utf-8. Error: "error":"code":"Gen1ApiDisabled","message":"Gen1 API are currently disabled...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.