Giter VIP home page Giter VIP logo

azure-databricks-api's Introduction

Azure Databricks API Wrapper

A Python, object-oriented wrapper for the Azure Databricks REST API 2.0

GitHub Workflow Status (branch) Coveralls github PyPI PyPI - Downloads GitHub

Installation

This package is pip installable.

pip install azure-databricks-api

Implemented APIs

As of June 25th, 2020 there are 12 different services available in the Azure Databricks API. Currently, the following services are supported by the Azure Databricks API Wrapper.

  • Clusters
  • Cluster Policies (Preview)
  • DBFS
  • Groups (Must be Databricks admin)
  • Instance Pools
  • Jobs
  • Libraries
  • MLflow
  • SCIM (Preview)
  • Secrets
  • Token
  • Workspace

Client Instantiation

To create the client object, you pass the Azure region your workspace is located in and the generated Personal Access Token

from azure_databricks_api import AzureDatabricksRESTClient

azure_region = '[INSERT YOUR AZURE REGION HERE]'
token = '[INSERT YOUR PERSONAL ACCESS TOKEN]' 

client = AzureDatabricksRESTClient(region=azure_region, token=token)

Clusters Client Usage

The services above are implemented as children objects of the client. For example, to pin a cluster, you can either pass the cluster_name or cluster_id to the pin() method:

client.clusters.pin(cluster_name='test_cluster_name')

The other services are implemented similarly. (e.g. client.tokens or client.groups)

azure-databricks-api's People

Contributors

croc541 avatar dbrooks-in avatar dbrooks5 avatar ezwiefel avatar nutipa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

azure-databricks-api's Issues

Jobs API Implementation

HI Just wanted to ask,
Was there something, too difficult to implement in the Jobs API and was therefore not implemented or is it just not implemented because of time constraints.

clusters.get function raises error if no clusters in workspace.

client.clusters.get() function raises error if no clusters in workspace.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<command-915677376113286> in <module>()
     17 
     18   try:
---> 19     client.clusters.get(cluster_name)
     20     print("  Cluster '{0}' already exists".format(cluster_name))
     21   except ResourceDoesNotExist:

/databricks/python/lib/python3.5/site-packages/azure_databricks_api/__clusters.py in get(self, cluster_name, cluster_id)
    355                                                   api_path=API_PATH,
    356                                                   cluster_name=cluster_name,
--> 357                                                   cluster_id=cluster_id)
    358 
    359     def pin(self, cluster_name=None, cluster_id=None):

/databricks/python/lib/python3.5/site-packages/azure_databricks_api/__clusters.py in __send_cluster_id_to_endpoint(self, method, api_path, cluster_name, cluster_id)
    200 
    201         if cluster_name and not cluster_id:
--> 202             cluster_id = self.get_cluster_id(cluster_name)
    203 
    204         data = {"cluster_id": cluster_id}

/databricks/python/lib/python3.5/site-packages/azure_databricks_api/__clusters.py in get_cluster_id(self, cluster_name)
    312         found_clusters = [
    313             ClusterInfo(id=cluster['cluster_id'], state=cluster['state'], start_time=cluster['start_time'])
--> 314             for cluster in clusters if cluster['cluster_name'] == cluster_name]
    315 
    316         if len(found_clusters) == 0:

TypeError: 'NoneType' object is not iterable

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.