As mentioned in #41, some functions don't require authentication against the Azure Service Manager. Specifically, blob storage functions will work given only the storage key.

By supporting a version of the function that does not enforce supplying the activeAzureContext argument, we can improve ease-of-use for certain use cases.

I will create a feature branch to test this functionality, prioritizing the following functions:

• azureListStorageBlobs()
• azureGetBlob()`

Add tests to:

• add a container
• put a blob
• get a blob
• delete a blob
• delete a container

To better fit with accepted coding practise in R and Microsoft, use lowerCamelCase() function names and arguments.

The github repo is AzureSMR but the package itself is AzureSM.

For consistency, should we rename one of these to match?

To help usability change:

TID -> tenantID
CID -> clientID
KEY -> authKey

In addition, return an empty data frame (or appropriate object), instead of NULL.

The two vignettes need updating to reflect the new naming convention

SAI reference not found

In the installation tutorial, the library loaded is still AzureSM instead of AzureSMR.

After upgrading to R 3.3.3 and curl 2.3, my code stopped working with the error below:

sc <- createAzureContext(tenantID = tenant$id, clientID = app$id, authKey = app$secret)
Error in curl::curl_fetch_memory(url, handle = handle) :
Problem with the SSL CA cert (path? access rights?)

The VM functions do not provide descriptive error handling when things go wrong.

VM Stop Async mode causes error

Some VMS have osType returning as NULL causing an error. Manage NULL values and avoid error

We have these blob store functions:

Other blob store functions: azureBlobCD, azureBlobLS, azureGetBlob, azureListStorageBlobs, azurePutBlob

If we don't have this function, should we expose the API?

The Authentication Key generated by Azure AD sometimes contains reserved characters which causes a 401 error when authenticated when passed in the authentication body

Validated error using Key <- URLencode(Key, reserved = TRUE) which then resulted in a successful 200.

Add URLencode by default into the Authenticate function

This is more of a feature request, but it would be great to be able to suppress messages from azureGetBlob (maybe via the existing verbose parameter). If the messages stay, it would be nice to append a new line character to the current message (see cat).

For example, this happens in azureCreateStorageContainer(). In this case, catch 409 return codes and return "OK".

A few functions were renamed incorrectly:

AzureSMR::azureblobCD
AzureSMR::azureblobFind
AzureSMR::azureCreateresourceGroup
AzureSMR::azureCreateSAcontainer
AzureSMR::azureCreatestorageAccount
AzureSMR::azureDeleteresourceGroup
AzureSMR::azureDeleteSAcontainer
AzureSMR::azureGetblob
AzureSMR::azureListSAblobs
AzureSMR::azureListSAcontainers
AzureSMR::azurePutblob

Some API function call return an xml_document. When this happens, properly display the relevant error information.

Many functions contain arguments that are no longer used, e.g:

• azToken

Remove these arguments

What does the "R" in the status codes mean? It's not listed.

> azureStopVM(azureActiveContext=sc, resourceGroup=RG, vmName=vmnames[1])
azureStopVM: Request Submitted:  2016-12-22 09:10:37
Updating(U), deallocating(D), starting(S), Stopped/Deallocated(-) 
**R**DDDDDDDDDDDDDDDDDDDD
Finished Deallocated Sucessfully:  2016-12-22 09:12:20
Finished:  2016-12-22 09:12:20
[1] "Done"

Do we need to return "Done"? Just a silent TRUE might be okay. This probably applies across many other of the functions too.

Similarly - what is A in the following:

azureDeployTemplate: Request Submitted:  2016-12-22 11:26:14
Running(R), Succeeded(S)
**A**RRRR

From R CMD check:

Found the following significant warnings:
  Warning: replacing previous import 'curl::handle_reset' by 'httr::handle_reset' 
  when loading 'AzureSM'

and:

checking dependencies in R code ... WARNING
'::' or ':::' import not declared from: 'RCurl'

This will make refactoring of the code easier.

Create a function:

callAzureStorageApi <- function(url, verb = "GET", storageKey, storageAccount,
                   headers = NULL, container = NULL, CMD, size = NULL, contenttype = NULL,
                   verbose = FALSE)

The returned JSON code contains the error code as well as more information about the error.

Create a helper function to print the full error message.

The function AzureListAllRecources() has a typo, and has been replaced with AzureListAllResources().

List the function appropriately as defunct.

Thanks for the nice AzureSMR package,

Was trying to use it with an Shiny application together with the Windows taskscheduler to start and stop
Azure VM.

Although due to the fact that we're using next to

• Microsoft.Compute/virtualMachines

also

• Microsoft.ClassicCompute/virtualMachines and
• Microsoft.ClassicCompute/domainNames

which we want also to restart on a scheduled way.

Is it possible to add as extra argumnent with these functions the type containing
one of the next types:

• Microsoft.Compute/virtualMachines <- default
• Microsoft.ClassicCompute/virtualMachines
• Microsoft.ClassicCompute/domainNames

and so the next code should be
instead of:

    URL <- paste("https://management.azure.com/subscriptions/", 
        SUBIDI, "/resourceGroups/", RGI, "/providers/Microsoft.Compute/virtualmachines/", 
        vmNameI, "/deallocate?api-version=2015-05-01-preview", 

   URL <- paste("https://management.azure.com/subscriptions/", 
        SUBIDI, "/resourceGroups/", RGI, "/providers/',type,"/", 
        vmNameI, "/deallocate?api-version=2015-05-01-preview", 

Or provide me feedback when this isn't a good idea

For naming consistency

Use the S3 class system to make it easier to view and validate the azureActiveContext object

Steps to repro:

1. Set valid values for the parameters in the code snippet below
2. Run the code snippet
   Result: Error message occurs
3. Run the SetAzureContext and dirlist (azureBlobLS) commands again
   Result: The directory list is retrieved

Error message:

> Error: 
Fetching Storage Key..
Fetching Storage Key..
Error in AzureSMR::azureBlobLS(sc, directory = "", recursive = FALSE,  : 
  Error: No storageKey provided: Use storageKey argument or set in AzureContext

Code snippet:

sc <- AzureSMR::createAzureContext();
AzureSMR::setAzureContext(sc,
                          tenantID = tenantID,
                          clientID =clientid,
                          authKey=authKey,
                          subscriptionID=subID, 
                          resourceGroup= rG, 
                          storageKey = storageKey, 
                          storageAccount = storageAccount);
AzureSMR::azureAuthenticate(sc);
AzureSMR::azureCheckToken(sc);
dirlist <- AzureSMR::azureBlobLS(sc, directory= "", recursive = FALSE,container = "input");

Fix typos, add seealso sections, etc.

Multiple issues:

• Many functions have arguments that are not documented.
• Some functions don't have descriptions documented properly.

In addition, documentation can be simplified by using inherits clauses

Use formatr to make the code style consistent:

• Use spaces after commas
• Add line breaks after ~80 characters
• Use {} consistently

The function azureListStorageBlobs() currently returns a maximum of 5000 records, even if more blobs exist. It would be useful to expose the marker and maxresults URI parameters for the "List Blobs" Blob Service API call (https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/list-blobs) so that all blobs can be listed. Exposing the prefix and delimiter parameters would also be helpful.

Use testthat to test package functionality

To simplify AzureSMR setup, combine Context and Authenticate operations

Microsoft policy prefers the MIT license over GPL, so unless there's a good reason and we have CELA approval for using GPL-3, we should change this.

dumpAzureContext(sc) causes an error rather than the deprecation message

Typing just sc does not print all of the variables.

Add some code examples.

Without code examples, R CMD check throws warnings.

Although almost all the code requires authentication, we can still use \dontrun{} segments to illustrate typical patterns.

I'm trying to use Azure Blob storage as follows but I'm getting an authentication error.:

Server failed to authenticate the request. 
Make sure the value of Authorization header is formed correctly including the signature

What am I doing wrong here?

sc <- createAzureContext(tenantID = "{TID}", clientID = "{CID}", authKey= "{KEY}")
sKey <- azureSAGetKey(sc, resourceGroup = "bnosac-training", storageAccount = "bnosacdata4training")

Fetching Storage Key..

azureListStorageContainers(sc, storageAccount = "bnosacdata4training", storageKey = sKey)

No encoding supplied: defaulting to UTF-8.
No encoding supplied: defaulting to UTF-8.
Error: azureListStorageContainers()
AuthenticationFailed
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:89983872-0001-00a6-1ccb-bdbf26000000
Time:2017-04-25T13:51:32.9974238Z
Return code: 403

From R CMD check:

AzureBlobCD: no visible binding for global variable 'sc'
AzureBlobFind: no visible binding for global variable 'sc'
AzureBlobLS: no visible binding for global variable 'sc'
AzureCreateHDI: no visible binding for global variable 'StorageAcc'
AzureCreateSAContainer: no visible binding for global variable 'sc'
AzureDeleteSAContainer: no visible binding for global variable 'sc'
AzureGetBlob: no visible binding for global variable 'sc'
AzureListSABlobs: no visible binding for global variable 'sc'
AzureListSAContainers: no visible binding for global variable 'sc'
AzurePutBlob: no visible binding for global variable 'sc'
Undefined global functions or variables:
  StorageAcc sc

I've read in a csv file (tmpFile), and now want to write that out to blob. Running this:
azurePutBlob(sc, blob = "tsv.txt", file = tmpFile)

Returns:
[1] "blob: directory/tsv.txt Saved: 1 bytes written"

And if I look at the blob file, it has just 1 character in it: '-'.
Looking at source, there is no handling of the file param. If contents is blank, it gets that character '-', and thats it.

I also tried:
azurePutBlob(sc, blob = "tsv.txt", contents = tmpFile)
And it run for awhile, looks like its uploading the data:
[1] "blob: directory/tsv.txt Saved: 545 bytes written" "blob: directory/tsv.txt Saved: 545 bytes written" [3] "blob: directory/tsv.txt Saved: 699 bytes written" "blob: directory/tsv.txt Saved: 545 bytes written" [5] "blob: directory/tsv.txt Saved: 545 bytes written" "blob: directory/tsv.txt Saved: 706 bytes written"

it goes up to [37] but then this is output and it stops uploading and the blob is never created:
Warning messages:
1: In if (nchar(contents) == 0) contents <- "-" :
the condition has length > 1 and only the first element will be used
2: In charToRaw(object) :
argument should be a character vector of length 1
all but the first element will be ignored

Description of SAS:

https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-shared-access-signature-part-1

R CMD check results:

checking package dependencies ... ERROR
Namespace dependency not required: 'digest'

Depends: includes the non-default packages:
  'httr' 'jsonlite' 'XML' 'plyr' 'base64enc' 'curl'
Adding so many packages to the search path is excessive and importing
selectively is preferable.

Add this to readme.md:

## Code of conduct

This project has adopted the [Microsoft Open Source Code of Conduct](https://microsoft.github.io/codeofconduct). For more information see the [Code of Conduct FAQ](https://microsoft.github.io/codeofconduct/faq.md) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.

Example

azureRunScriptAction(sc, 
                     scriptname = "installPackages",
                     scriptURL = "http://mrsactionscripts.blob.core.windows.net/rpackages-v01/InstallRPackages.sh",
                     workerNode = TRUE, edgeNode = TRUE,
                     parameters = "useCRAN stringr")

Returns:

Error in azureRunScriptAction(... 
  Error: Return code 404

Possible Bug Location

It looks like the issue is in the following httr call:

  r <- POST(URL, add_headers(.headers = c(Host = "management.azure.com", 
    Authorization = ATI, `Content-type` = "application/json")), 
    body = bodyI, encode = "json", verbosity)

Currrently only supports HDI Standard.
Workaround - use templates.

Plan :- add support for HDI Premium + ADLS

Seems to relate to ID now being called id

Many functions contain cat() statements to print to the console. This is messy, and should be replace with message() to separate messages from actual program output.