Giter VIP home page Giter VIP logo

cosmosdb's Introduction

License Documentation - CosmosDB PowerShell Gallery - CosmosDB Minimum Supported PowerShell Version Minimum Supported PowerShell Core Version

CosmosDB PowerShell Module

Module Build Status

Branch Build Status Code Coverage
dev Build status codecov
master Build status codecov

Table of Contents

Introduction

This PowerShell module provides cmdlets for working with Azure Cosmos DB.

The CosmosDB PowerShell module enables management of:

The module uses the CosmosDB (DocumentDB) Rest APIs.

For more information on the CosmosDB Rest APIs, see this link.

Requirements

This module supports the following:

  • Windows PowerShell 5.x:
    • AzureRM.Profile and AzureRM.Resources PowerShell modules are required if using New-CosmosDbContext -ResourceGroup $resourceGroup

or:

  • PowerShell Core 6.x:
    • AzureRM.NetCore.Profile and AzureRM.NetCore.Resources PowerShell modules are required if using New-CosmosDbContext -ResourceGroup $resourceGroup

Installation

To install the module from PowerShell Gallery, use the PowerShell Cmdlet:

Install-Module -Name CosmosDB

Getting Started

The easiest way to use this module is to first create a context object using the New-CosmosDbContext cmdlet which you can then use to pass to the other CosmosDB cmdlets in the module.

To create the context object you will either need access to the primary primary or secondary keys from your CosmosDB account or allow the CosmosDB module to retrieve the keys directly from the Azure management portal for you.

Working with Contexts

Create a Context specifying the Key Manually

First convert your key into a secure string:

$primaryKey = ConvertTo-SecureString -String 'GFJqJesi2Rq910E0G7P4WoZkzowzbj23Sm9DUWFX0l0P8o16mYyuaZBN00Nbtj9F1QQnumzZKSGZwknXGERrlA==' -AsPlainText -Force

Use the key secure string, Azure CosmosDB account name and database to create a context variable:

$cosmosDbContext = New-CosmosDbContext -Account 'MyAzureCosmosDB' -Database 'MyDatabase' -Key $primaryKey

Use CosmosDB Module to Retrieve Key from Azure Management Portal

To create a context object so that the CosmosDB module retrieves the primary or secondary key from the Azure Management Portal, use the following command:

$cosmosDbContext = New-CosmosDbContext -Account 'MyAzureCosmosDB' -Database 'MyDatabase' -ResourceGroup 'MyCosmosDbResourceGroup' -MasterKeyType 'SecondaryMasterKey'

Note: if PowerShell is not connected to Azure then an interactive Azure login will be initiated. If PowerShell is already connected to an account that doesn't contain the CosmosDB you wish to connect to then you will first need to connect to the correct account using the Add-AzureRmAccount cmdlet.

Create a Context for a CosmosDB Emulator

Microsoft provides a CosmosDB emulator that you can run locally to enable testing and debugging scenarios. To create a context for a CosmosDB emulator installed on the localhost use the following command:

$cosmosDbContext = New-CosmosDbContext -Emulator -Database 'MyDatabase'

Create a Context from Resource Authorization Tokens

See the section Using Resource Authorization Tokens for instructions on how to create a Context object containing one or more Resource Authorization Tokens.

Working with Databases

Create a new database in the CosmosDB account:

New-CosmosDbDatabase -Context $cosmosDbContext -Id 'AnotherDatabase'

Get a list of databases in the CosmosDB account:

Get-CosmosDbDatabase -Context $cosmosDbContext

Get the specified database from the CosmosDB account:

Get-CosmosDbDatabase -Context $cosmosDbContext -Id 'MyDatabase'

Working with Offers

Get a list of offers in the CosmosDB account:

Get-CosmosDbOffer -Context $cosmosDbContext

Query the offers in the CosmosDB account:

Get-CosmosDbOffer -Context $cosmosDbContext -Query 'SELECT * FROM root WHERE (root["id"] = "lyiu")'

Update an existing V2 offer to set a different throughput:

Get-CosmosDbOffer -Context $cosmosDbContext -Id 'lyiu' |
    Set-CosmosDbOffer -Context $cosmosDbContext -OfferThroughput 1000 -OfferIsRUPerMinuteThroughputEnabled $true

Update all existing V2 offers to set a different throughput:

Get-CosmosDbOffer -Context $cosmosDbContext -Query 'SELECT * FROM root WHERE (root["offerVersion"] = "V2")' |
    Set-CosmosDbOffer -Context $cosmosDbContext -OfferThroughput 10000 -OfferIsRUPerMinuteThroughputEnabled $false

Working with Collections

Get a list of collections in a database:

Get-CosmosDbCollection -Context $cosmosDbContext

Create a collection in the database with the offer throughput of 2500 RU/s:

New-CosmosDbCollection -Context $cosmosDbContext -Id 'MyNewCollection' -OfferThroughput 2500

Create a collection in the database with the partition key 'account' and the offer throughput of 50000 RU/s:

New-CosmosDbCollection -Context $cosmosDbContext -Id 'PartitionedCollection' -PartitionKey 'account' -OfferThroughput 50000

Get a specified collection from a database:

Get-CosmosDbCollection -Context $cosmosDbContext -Id 'MyNewCollection'

Delete a collection from the database:

Remove-CosmosDbCollection -Context $cosmosDbContext -Id 'MyNewCollection'

Creating a Collection with a custom Indexing Policy

You can create a collection with a custom indexing policy by assembling an Indexing Policy object using the functions:

  • New-CosmosDbCollectionIncludedPathIndex
  • New-CosmosDbCollectionIncludedPath
  • New-CosmosDbCollectionExcludedPath
  • New-CosmosDbCollectionIndexingPolicy

For example, to create a string range, a number range index and a point spatial index on the '/*' path using consistent indexing mode with no excluded paths:

$indexStringRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType String -Precision -1
$indexNumberRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType Number -Precision -1
$indexNumberSpatial = New-CosmosDbCollectionIncludedPathIndex -Kind Spatial -DataType Point
$indexIncludedPath = New-CosmosDbCollectionIncludedPath -Path '/*' -Index $indexStringRange, $indexNumberRange, $indexNumberSpatial
$indexingPolicy = New-CosmosDbCollectionIndexingPolicy -Automatic $true -IndexingMode Consistent -IncludedPath $indexIncludedPath
New-CosmosDbCollection -Context $cosmosDbContext -Id 'MyNewCollection' -PartitionKey 'account' -IndexingPolicy $indexingPolicy

For more information on how CosmosDB indexes documents, see this page.

Update an existing Collection with a new Indexing Policy

You can update an existing collection with a custom indexing policy by assembling an Indexing Policy using the method in the previous section and then applying it using the Set-CosmosDbCollection function:

$indexStringRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType String -Precision -1
$indexIncludedPath = New-CosmosDbCollectionIncludedPath -Path '/*' -Index $indexStringRange
$indexingPolicy = New-CosmosDbCollectionIndexingPolicy -Automatic $true -IndexingMode Consistent -IncludedPath $indexIncludedPath
Set-CosmosDbCollection -Context $cosmosDbContext -Id 'MyNewCollection' -IndexingPolicy $indexingPolicy

Working with Documents

Create 10 new documents in a collection in the database:

0..9 | Foreach-Object {
    $document = @"
{
    `"id`": `"$([Guid]::NewGuid().ToString())`",
    `"content`": `"Some string`",
    `"more`": `"Some other string`"
}
"@
New-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -DocumentBody $document
}

Get the first 5 documents from the collection in the database:

$resultHeaders = ''
$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -MaxItemCount 5 -ResultHeaders $resultHeaders
$continuationToken = $resultHeaders.value.'x-ms-continuation'

Get the next 5 documents from a collection in the database using the continuation token found in the headers from the previous request:

$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -MaxItemCount 5 -ContinuationToken $continuationToken

Replace the content of a document in a collection in the database:

$newDocument = @"
{
    `"id`": `"$($documents[0].id)`",
    `"content`": `"New string`",
    `"more`": `"Another new string`"
}
"@
Set-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id $documents[0].id -DocumentBody $newDocument

Return a document with a specific Id from a collection in the database:

Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id $documents[0].id

Querying a collection in a database:

$query = "SELECT * FROM customers c WHERE (c.id = '[email protected]')"
Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Query $query

Querying a collection in a database using a parameterized query:

$query = "SELECT * FROM customers c WHERE (c.id = @id)"
$queryParameters = @(
    @{ name = "@id"; value="[email protected]"; }
)
Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Query $query -QueryParameters $queryParameters

Delete a document from a collection in the database:

Remove-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id $documents[0].id

Working with Documents in a Partitioned Collection

Creating a document in a collection that has a Partition Key requires the PartitionKey parameter to be specified for the document:

$document = @"
{
    `"id`": `"en-us`",
    `"locale`": `"English (US)`"
}
"@
New-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'PartitionedCollection' -DocumentBody $document -PartitionKey 'en-us'

Get a document from a partitioned collection with a specific Id:

$document = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'PartitionedCollection' -Id 'en-us' -PartitionKey 'en-us'

Delete a document from a partitioned collection in the database:

Remove-CosmosDbDocument -Context $cosmosDbContext -CollectionId 'PartitionedCollection' -Id 'en-us' -PartitionKey 'en-us'

Working with Attachments

Create an attachment on a document in a collection:

New-CosmosDbAttachment -Context $cosmosDbContext -CollectionId 'MyNewCollection' -DocumentId $documents[0].id -Id 'image_1' -ContentType 'image/jpg' -Media 'www.bing.com'

Get all attachments for a document in a collection:

Get-CosmosDbAttachment -Context $cosmosDbContext -CollectionId 'MyNewCollection' -DocumentId $documents[0].id

Get an attachment by Id for a document in a collection:

Get-CosmosDbAttachment -Context $cosmosDbContext -CollectionId 'MyNewCollection' -DocumentId $documents[0].id -Id 'image_1'

Rename an attachment for a document in a collection:

Set-CosmosDbAttachment -Context $cosmosDbContext -CollectionId 'MyNewCollection' -DocumentId $documents[0].id -Id 'image_1' -NewId 'Image_2'

Delete an attachment from a document in collection:

Remove-CosmosDbAttachment -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id $documents[0].id -Id 'Image_2'

Working with Users

Get a list of users in the database:

Get-CosmosDbUser -Context $cosmosDbContext

Create a user in the database:

New-CosmosDbUser -Context $cosmosDbContext -Id '[email protected]'

Delete a user from the database:

Remove-CosmosDbUser -Context $cosmosDbContext -Id '[email protected]'

Working with Permissions

Get a list of permissions for a user in the database:

Get-CosmosDbPermission -Context $cosmosDbContext -UserId '[email protected]'

Create a permission for a user in the database with read access to a collection:

$collectionId = Get-CosmosDbCollectionResourcePath -Database 'MyDatabase' -Id 'MyNewCollection'
New-CosmosDbPermission -Context $cosmosDbContext -UserId '[email protected]' -Id 'r_mynewcollection' -Resource $collectionId -PermissionMode Read

Remove a permission for a user from the database:

Remove-CosmosDbPermission -Context $cosmosDbContext -UserId '[email protected]' -Id 'r_mynewcollection'

Using Resource Authorization Tokens

Cosmos DB supports using resource authorization tokens to grant access to individual resources (eg. documents, collections, triggers) to a specific user. A user in this context can also be used to represent an application that needs access to specific data. This can be used to reduce the need to provide access to master keys to end users.

To use a resource authorization token, first a permission must be assigned to the user for the resource using the New-CosmosDbPermission. A user can be created using the New-CosmosDbUser function.

Note: By default, Resource Authorization Tokens expire after an hour. This can be extended to a maximum of 5 hours or reduced to minimum of 10 minutes. Use the TokenExpiry parameter to control the length of time that the resource authorization tokens will be valid for.

The typical pattern for using resource authorization tokens is to have a token broker app that provides some form of user authentication and then returns the resource authorization tokens assigned to that user. This removes the requirement for the user to be given access to the master key for the CosmosDB database.

For more information on using resource authorization tokens or the *token broker app pattern, please see this document.

The following is an example showing how to create a resource context object that contains a resource authorization token granting access to read the collection MyNewCollection. It is assumed that the permission for the user [email protected] has been created as per the previous section. The resource context object is then used to retrieve the MyNewCollection.

The resource authorization token is stored in the context object with an expiration date/time matching what was returned in the permission so that the validity of a token can be validated and reported on without making a request to the Cosmos DB server.

$collectionId = Get-CosmosDbCollectionResourcePath -Database 'MyDatabase' -Id 'MyNewCollection'
$permission = Get-CosmosDbPermission -Context $cosmosDbContext -UserId '[email protected]' -Id 'r_mynewcollection' -Resource $collectionId -TokenExpiry 7200
# Future features planned to make creation of a resource context token from a permission easier
$contextToken = New-CosmosDbContextToken `
    -Resource $collectionId `
    -TimeStamp $permission[0].Timestamp `
    -TokenExpiry 7200 `
    -Token (ConvertTo-SecureString -String $permission[0].Token -AsPlainText -Force)
$resourceContext = New-CosmosDbContext `
    -Account $cosmosDBContext.Account
    -Database 'MyDatabase' `
    -Token $contextToken
Get-CosmosDbCollection `
    -Context $resourceContext `
    -Id 'MyNewCollection' `

Working with Triggers

Get a list of triggers for a collection in the database:

Get-CosmosDbTrigger -Context $cosmosDbContext -CollectionId 'MyNewCollection'

Create a trigger for a collection in the database that executes after all operations:

$body = @'
function updateMetadata() {
    var context = getContext();
    var collection = context.getCollection();
    var response = context.getResponse();
    var createdDocument = response.getBody();

    // query for metadata document
    var filterQuery = 'SELECT * FROM root r WHERE r.id = "_metadata"';
    var accept = collection.queryDocuments(collection.getSelfLink(), filterQuery, updateMetadataCallback);
    if(!accept) throw "Unable to update metadata, abort";

    function updateMetadataCallback(err, documents, responseOptions) {
        if(err) throw new Error("Error" + err.message);

        if(documents.length != 1) throw 'Unable to find metadata document';
        var metadataDocument = documents[0];

        // update metadata
        metadataDocument.createdDocuments += 1;
        metadataDocument.createdNames += " " + createdDocument.id;

        var accept = collection.replaceDocument(metadataDocument._self, metadataDocument, function(err, docReplaced) {
            if(err) throw "Unable to update metadata, abort";
        });

        if(!accept) throw "Unable to update metadata, abort";
        return;
    }
}
'@
New-CosmosDbTrigger -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'MyTrigger' -TriggerBody $body -TriggerOperation All -TriggerType Post

Update an existing trigger for a collection in the database to execute before all operations:

$body = @'
function updateMetadata() {
    var context = getContext();
    var collection = context.getCollection();
    var response = context.getResponse();
    var createdDocument = response.getBody();

    // query for metadata document
    var filterQuery = 'SELECT * FROM root r WHERE r.id = "_metadata"';
    var accept = collection.queryDocuments(collection.getSelfLink(), filterQuery, updateMetadataCallback);
    if(!accept) throw "Unable to update metadata, abort";

    function updateMetadataCallback(err, documents, responseOptions) {
        if(err) throw new Error("Error" + err.message);

        if(documents.length != 1) throw 'Unable to find metadata document';
        var metadataDocument = documents[0];

        // update metadata
        metadataDocument.createdDocuments += 1;
        metadataDocument.createdNames += " " + createdDocument.id;

        var accept = collection.replaceDocument(metadataDocument._self, metadataDocument, function(err, docReplaced) {
            if(err) throw "Unable to update metadata, abort";
        });

        if(!accept) throw "Unable to update metadata, abort";
        return;
    }
}
'@
Set-CosmosDbTrigger -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'MyTrigger' -Body $body -TriggerOperation All -TriggerType Pre

Remove a trigger for a collection from the database:

Remove-CosmosDbTrigger -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'MyTrigger'

Working with Stored Procedures

Get a list of stored procedures for a collection in the database:

Get-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId 'MyNewCollection'

Create a stored procedure for a collection in the database:

$body = @'
function () {
    var context = getContext();
    var response = context.getResponse();

    response.setBody("Hello, World");
}
'@
New-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'spHelloWorld' -StoredProcedureBody $body

Update an existing stored procedure for a collection in the database:

$body = @'
function (personToGreet) {
    var context = getContext();
    var response = context.getResponse();

    response.setBody("Hello, " + personToGreet);
}
'@
Set-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'spHelloWorld' -StoredProcedureBody $body

Execute a stored procedure for a collection from the database:

Invoke-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'spHelloWorld' -StoredProcedureParameters @('PowerShell')

Remove a stored procedure for a collection from the database:

Remove-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'spHelloWorld'

Working with User Defined Functions

Get a list of user defined functions for a collection in the database:

Get-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId 'MyNewCollection'

Create a user defined function for a collection in the database:

$body = @'
function tax(income) {
    if(income == undefined) throw 'no input';
    if (income < 1000)
        return income * 0.1;
    else if (income < 10000)
        return income * 0.2;
    else
        return income * 0.4;
}
'@
New-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'udfTax' -UserDefinedFunctionBody $body

Update an existing user defined function for a collection in the database:

$body = @'
function tax(income) {
    if(income == undefined) throw 'no input';
    if (income < 1000)
        return income * 0.2;
    else if (income < 10000)
        return income * 0.3;
    else
        return income * 0.4;
}
'@
Set-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'udfTax' -Body $body

Remove a user defined function for a collection from the database:

Remove-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId 'MyNewCollection' -Id 'udfTax'

How to Handle Exceeding Provisioned Throughput

When using Azure Cosmos DB it is quite common to exceed the throughput that has been provisioned against a collection (or accross multiple collections). See this page for more information on request units and throughput provisioning.

When this happens requests will return a Too Many Request (error code 429). Usually just waiting a small amount of time and trying again will result in the request succeeding. However, the Cosmos DB PowerShell module provides a mechanism for configuring an automatic back-off and retry policy.

This is configured within the Context object that is usually passed to each Cosmos DB module function.

To configure a Back-off Policy, use the New-CosmosDbBackoffPolicy function:

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 5
$cosmosDbContext = New-CosmosDbContext -Account 'MyAzureCosmosDB' -Database 'MyDatabase' -Key $primaryKey -BackoffPolicy $backoffPolicy

This will cause any functions that use the Context to automatically retry up to 5 times if a 429 response code is returned. Any other type of response code will throw an exception. The number of milliseconds to delay before retrying will be determined automatically by using the x-ms-retry-after-ms header returned by Cosmos DB.

Additional Back-off Policy options can be set to override or extend the value returned in the x-ms-retry-after-ms header.

Note: if the delay calculated by the policy is less than the value returned in the x-ms-retry-after-ms header, then the x-ms-retry-after-ms value will always be used.

The available Back-off Methods are:

  • Default
  • Additive
  • Linear
  • Exponential
  • Random

The following show examples of alternative policy back-off types that can implemented:

Default

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 10 -Method Default -Delay 100

The delay of 100ms will always be used unless it is less than x-ms-retry-after-ms. The delay can be set to 0 and will cause the x-ms-retry-after-ms to always be used. It is the default Back-off Policy behavior.

Additive

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 10 -Method Additive -Delay 1000

This will create a policy that will retry 10 times with a delay equaling the value of the returned x-ms-retry-after-ms header plus 1000ms.

Linear

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 3 -Method Linear -Delay 500

This will create a policy that will wait for 500ms on the first retry, 1000ms on the second retry, 1500ms on final retry.

Exponential

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 4 -Method Exponential -Delay 1000

This will create a policy that will wait for 1000ms on the first retry, 4000ms on the second retry, 9000ms on the 3rd retry and 16000ms on the final retry.

Random

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 3 -Method Random -Delay 1000

A policy that adds or subtracts up to 50% of the delay period to the base delay each time can also be applied. For example, the first delay might be 850ms, with the second delay being 1424ms and final delay being 983ms.

Contributing

If you wish to contribute to this project, please read the Contributing.md document first. We would be very grateful of any contributions.

Cmdlets

A list of Cmdlets in the CosmosDB module can be found by running the following PowerShell commands:

Import-Module CosmosDB
Get-Command -Module CosmosDB

Help on individual Cmdlets can be found in the built-in Cmdlet help:

Get-Help -Name Get-CosmosDBUser

The details of the cmdlets contained in this module can also be found in the wiki.

Change Log

For a list of changes to versions, see the CHANGELOG.md file.

Links

cosmosdb's People

Contributors

bak-t avatar jasonchester avatar mwl88 avatar plagueho avatar smneal avatar watersjohn avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.