Currently there are several traits for indicating how to get the name of different resources. These are usually used in a dynamic dispatch way (e.g., fn foo(name: &dyn DatabaseName)). Since the only thing that these traits allow you to do is to get a string slice whose lifetime is tied to the borrow of the trait object, I don't really see a need for the trait object indirection. Can these trait objects be removed and simply replaced with string slices?

according to the documentation (https://docs.microsoft.com/en-us/rest/api/storageservices/get-blob-properties) a get-blob-properties call should contain the blobs metadata, but for me it always returns an empty hashmap. looking inside Blob::from_headers I do not see that the x-meta-name:value headers are read in there.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/clear-messages.

• The storage crate is not ideally structured. I simply moved the previous crates into a single crate with submodules. We might want to rearrange things.
• The storage crate defaults to all features. This is convenient, but means that by default the project will be slow to build. What's the right trade off here?
• Should we rename auth_aad to just auth and have AAD functionality behind a feature (so that if we add new auth varieties we can simply add new features?
• I've left azure_sdk_core named that way even though it contains sdk. I think azure_core is a bit too generic of a package name.

service name

I've been hoping to derive the service name from the Python package-name or output-folder. The package-name was looking like a better option, but let's review a couple of examples. In #34, two of our Rust sdk packages here were named azure_cosmos and azure_service_bus.

https://github.com/Azure/azure-rest-api-specs/blob/master/specification/cosmos-db/resource-manager/readme.python.md
package-name: azure-mgmt-cosmosdb
output-folder: $(python-sdks-folder)/cosmos/azure-mgmt-cosmosdb/azure/mgmt/cosmosdb

https://github.com/Azure/azure-rest-api-specs/blob/master/specification/cosmos-db/data-plane/readme.python.md
package-name: azure-table
output-folder: "$(python-sdks-folder)/sdk/cosmos/azure-cosmos-table/azure/table/_generated"

https://github.com/Azure/azure-rest-api-specs/blob/master/specification/servicebus/resource-manager/readme.md
package-name: azure-mgmt-servicebus
output-folder: $(python-sdks-folder)/servicebus/azure-mgmt-servicebus/azure/mgmt/servicebus

https://github.com/Azure/azure-rest-api-specs/blob/master/specification/storage/resource-manager/readme.python.md
package-name: azure-mgmt-storage
output-folder: $(python-sdks-folder)/storage/azure-mgmt-storage/azure/mgmt/storage/

After going through this, I think it is best to not rely on another SDK, but use the specification folder name with a map to rename those that we want to. In this case specification/cosmos-db would automatically map to cosmos_db, but we could have an entry to map it to cosmos instead. For specification/servicebus, we could add entry to map it to service_bus.

package name

We need to come up with naming for Azure control plane and data plane. "control plane" aka "resource-manager". In #31, I had the control plane crate name as azure_mgmt_${service}, then changed it to azure_mgmt_${service}, but now think it would be better to change it back. The .NET package name is Microsoft.Azure.Management.CosmosDB were Management comes before the service name. It is easy to query for all management packages in the NuGet Gallery. Similarly, the python package name is azure-mgmt-cosmosdb. What should our Rust control plane package be named:

• azure_mgmt_cosmosdb
• azure_mgmt_cosmos
• azure_mgmt_cosmos_db
• azure_cosmosdb_mgmt
• azure_cosmos_mgmt
• azure_cosmos_db_mgmt

For the data plane, I'm not really sure what to do yet. If we use azure_mgmt_ or azure_management_ as a prefix for the control plane, it would be nice to use a prefix like azure_data_, azure_svc_, or azure_service_ for the data plane. As an example, let's look at the Azure Data Lake Storage. First, let's look at the URL from marketing:
https://azure.microsoft.com/en-us/services/storage/data-lake-storage/
And actually, if we look at the marketing service name for "Azure Cosmos DB":
https://azure.microsoft.com/en-us/services/cosmos-db/

Here are the package names for Azure Data Lake Strorage data plane:
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/storage/data-plane/readme.md

• .NET: Microsoft.Azure.Storage.DataLake
• Python: azure-datalake-storage

What should the Rust data plane package name be?

• azure_datalake_storage
• azure_data_datalake_storage
• azure_svc_datalake_storage
• azure_service_datalake_storage
• azure_data_lake_storage
• azure_data_data_lake_storage
• azure_svc_data_lake_storage
• azure_service_data_lake_storage

Are crate names limited to 64 chars? That is a lot of characters, but may be something to keep in mind.
rust-lang/crates.io#718

The method borrows account and token parameters for an indefinite time.
This makes it harder for use. Is there any reason to that or the API can be adapted to work in the same way as client::with_access_key or client::with_azure_sas ?

As per https://docs.microsoft.com/en-us/rest/api/storageservices/preflight-queue-request.

The service examples currently get the access token and subscription ID from environment variables. It would be nice to get them from the AzureCliCredential.

A current example from #31:

/*
Create a resource group, similar to:
az group create --name $RESOURCE_GROUP_NAME --location $RESOURCE_GROUP_LOCATION

export SUBSCRIPTION_ID=$(az account show --query id --output tsv)
export ACCESS_TOKEN=$(az account get-access-token --query accessToken --output tsv)
export RESOURCE_GROUP_NAME=azuresdkforrust
export RESOURCE_GROUP_LOCATION=southcentralus
cargo run --example group_create
*/

use azure_mgmt_resources::{models::ResourceGroup, operations::resource_groups};

type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;

#[tokio::main]
async fn main() -> Result<()> {
    let access_token = &get_access_token()?;
    let subscription_id = &get_subscription_id()?;
    let resource_group_name = &get_resource_group_name()?;
    let resource_group_location = get_resource_group_location()?;
    let config = &azure_mgmt_resources::Configuration::new(access_token);

    let group = ResourceGroup {
        id: None,
        name: None,
        type_: None,
        properties: None,
        location: resource_group_location,
        managed_by: None,
        tags: None,
    };
    let group_created = resource_groups::create_or_update(config, resource_group_name, &group, subscription_id).await?;
    println!("group created: {:#?}", group_created);
    Ok(())
}

fn get_subscription_id() -> Result<String> {
    Ok(std::env::var("SUBSCRIPTION_ID").map_err(|_| "SUBSCRIPTION_ID required")?)
}

fn get_access_token() -> Result<String> {
    Ok(std::env::var("ACCESS_TOKEN").map_err(|_| "ACCESS_TOKEN required")?)
}

fn get_resource_group_name() -> Result<String> {
    Ok(std::env::var("RESOURCE_GROUP_NAME").map_err(|_| "RESOURCE_GROUP_NAME required")?)
}

fn get_resource_group_location() -> Result<String> {
    Ok(std::env::var("RESOURCE_GROUP_LOCATION").map_err(|_| "RESOURCE_GROUP_LOCATION required")?)
}

As per https://docs.microsoft.com/en-us/rest/api/storageservices/set-queue-metadata.

the put-block-list call supports setting this md5 hash that we see in Blob for when you get-blob it back (see https://docs.microsoft.com/en-us/rest/api/storageservices/put-block-list).

I wonder if you guys have a high level doc how to contribute? cause I am confused about what code generation is going on in here if I were to add this missing header myself?

We'd like to rearrange the crates in this repo to better reflect how the other SDKs are arranged since such an arrangement is still idiomatic Rust and is more consistent with existing SDKs.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/update-message.

TL;DR;
when is the new repo used for a crates release?

the current crates release is based on the old archived repo which refers to this repo here. confusingly this repo here then refers back to releases on crates.

now I am using the current crates releases:

azure_sdk_core = "0.43"
azure_sdk_storage_core = "0.44"
azure_sdk_storage_blob = "0.45"

to access the azure blob store. this is 3months old and lacks some api methods that are already implemented in here.

how are people using this new repo right now? direct git-ref?

My first thought was that resource should be a list of scopes instead to match Microsoft Authentication Library MSAL v2, but I'm not sure.

/// Represents a credential capable of providing an OAuth token.
#[async_trait::async_trait]
pub trait TokenCredential {
    /// Gets a `TokenResponse` for the specified resource
    async fn get_token(&self, resource: &str) -> Result<TokenResponse, AzureError>;
}

My guess is that resource here is modeled after he az command line and is the resource identifier. I think it becomes the scope query param. In MSAL v2, it is a space delimited list.

Links:

• https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az_account_get_access_token
• https://cran.r-project.org/web/packages/AzureAuth/vignettes/token.html
• https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-permissions-and-consent

Some crates have E2E tests in addition to local ones. These tests ensure the correctness of the implemented APIs by checking them against the Azure platform.
We have to decide if it makes sense to reenable these tests in the current CI pipeline.

PS: We can kick off the tests with cargo test --features=test_e2e in the supported modules.

As mentioned in #31. This can be reproduced with the cargo run --example vm_list from #31.

I think it is an important use case to be able to target wasm & wasi with these crates. We will need to add these targets to CI:

rustup target add wasm32-unknown-unknown
rustup target add wasm32-wasi

I'm hoping we can get the right features in place to support this. An important part of this is being able to use a different http client like reqwest instead of hyper. The http client abstraction is #36.

Currently we have an auth_aad crate, but authorization may take many forms. Additionally, it seems that most of the other language SDKs don't use this naming convention for their auth packagaes. JS and C# use "authorization", but go uses "azidentity". Shall we converge on the JS and C# naming conventions?

As correctly noted by Ben Sully in #112, not using a let binding in request builders makes futures not Sendable.

The PR fixes one builder but we should extend the fix to all the other builders.

ctaggart/autorust#42

Many apps that use these libraries will need to have the http connections pooled. The Client types in both reqwest and surf pool the http connections.

The Client holds a connection pool internally, so it is advised that you create one and reuse it.
https://docs.rs/reqwest/0.10.8/reqwest/struct.Client.html

Reuses connections through the Client interface
https://docs.rs/surf/2.1.0/surf/

So references to the http client need to be passed in to types in Core that do http requests. We may also need to look into seeing how to update dependencies for the say requirement. For example, this is creating a new Client every time:

• https://github.com/ramosbugs/oauth2-rs/blob/main/src/reqwest.rs

As per https://docs.microsoft.com/en-us/rest/api/storageservices/get-queue-acl.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/set-queue-acl.

Instead of relying on a hardcoded reqwest or hyper client, we should create an abstraction in the core crate that allows us to plug in different implementations if we want to. We need to decide how that plugability works. The choices I think we have are:

• A single concrete type with implementations decided on compile time feature flags.
• Make the Configuration struct generic over some type T that implements a Client trait
• Store a Box in the configuration struct.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/create-queue4.

As I see keyvault requires to create a cliantId + secret to access the the vault. According to my understanding the other authentication methods ex provided by DefaultCredential should be enough to access the vault.

It'b be great to have access to the keyvault using only the AD DefaultCredential (ex managed identity for prod, cli logins for dev, etc). in a similar way as in c#.

When I was hacking with the ad credentials (az cli credentail is not working on windows as it cannot find az for some reason) i found the all the members of TokenResponse are public but new is private. As one can construct it like TokenResponse{token: tr.access_token, expires_on:tr.expires_on} it makes not much sense to restrict the visibility of new

The generated code for each operation currently takes a Configuration. This is how opening-generator currently does it. The other Azure SDKs generate Clients instead. It may be possible to do it that way instead. We will need to evaluate the trade-offs.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/delete-message2.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/get-messages.

Part of this sdk migrated from failure to thiserror a few months ago. Based in large part on an article titled Migrating from quick-error to SNAFU: a story on revamped error handling in Rust and its pull request Enet4/dicom-rs#62, I'm looking at using snafu. I created a branch that switches the AutoRust code to use it. It is possible to just wrap the source error types at first, then come back later to add more specialized errors and context info when needed.

Here is an example of my plan a service operation. As an example, let's look at "operationId": "ResourceGroups_CreateOrUpdate",. It returns these responses:

        "responses": {
          "200": {
            "description": "OK - Returns information about the new resource group.",
            "schema": {
              "$ref": "#/definitions/ResourceGroup"
            }
          },
          "201": {
            "description": "Created - Returns information about the new resource group.",
            "schema": {
              "$ref": "#/definitions/ResourceGroup"
            }
          },
          "default": {
            "description": "Error response describing why the operation failed.",
            "schema": {
              "$ref": "#/definitions/CloudError"
            }
          }
        }

I am assuming for now that if successful responses are defined, that default is an error response. If that doesn't prove to be true, a DefaultResponse will need to be added. If there is more than one successful response, they can be wrapped in a Response enum. A new Error type is created to capture any defined error responses or additional errors. Both types defined in a module named the same as the operation function.

pub mod resource_groups {
    use crate::{models::*};
    use reqwest::StatusCode;
    use snafu::{ResultExt, Snafu};

    pub async fn create_or_update(
        configuration: &crate::Configuration,
        resource_group_name: &str,
        parameters: &ResourceGroup,
        subscription_id: &str,
    ) -> std::result::Result<create_or_update::Response, create_or_update::Error> {
        let client = &configuration.client;
        let uri_str = &format!(
            "{}/subscriptions/{}/resourcegroups/{}",
            &configuration.base_path, subscription_id, resource_group_name
        );
        let mut req_builder = client.put(uri_str);
        if let Some(token) = &configuration.bearer_access_token {
            req_builder = req_builder.bearer_auth(token);
        }
        req_builder = req_builder.query(&[("api-version", &configuration.api_version)]);
        req_builder = req_builder.json(parameters);
        let req = req_builder.build().context(create_or_update::BuildRequestError)?;
        let rsp = client.execute(req).await.context(create_or_update::ExecuteRequestError)?;
        match rsp.status() {
            StatusCode::OK => {
                let body = rsp.bytes().await.context(create_or_update::ResponseBytesError)?;
                let rsp_value: ResourceGroup = serde_json::from_slice(&body).context(create_or_update::DeserializeError { body })?;
                Ok(create_or_update::Response::Ok200(rsp_value))
            }
            StatusCode::CREATED => {
                let body = rsp.bytes().await.context(create_or_update::ResponseBytesError)?;
                let rsp_value: ResourceGroup = serde_json::from_slice(&body).context(create_or_update::DeserializeError { body })?;
                Ok(create_or_update::Response::Created201(rsp_value))
            }
            status_code => {
                let body = rsp.bytes().await.context(create_or_update::ResponseBytesError)?;
                let rsp_value: CloudError = serde_json::from_slice(&body).context(create_or_update::DeserializeError { body })?;
                create_or_update::DefaultErrorResponse { status_code, value: rsp_value }.fail()
            }
        }
    }

    pub mod create_or_update {
        use crate::models::*;
        use reqwest::StatusCode;
        use snafu::Snafu;

        pub enum Response {
            Ok200(ResourceGroup),
            Created201(ResourceGroup),
        }

        #[derive(Debug, Snafu)]
        #[snafu(visibility(pub(crate)))]
        pub enum Error {
            DefaultErrorResponse { status_code: StatusCode, value: CloudError },
            BuildRequestError { source: reqwest::Error },
            ExecuteRequestError { source: reqwest::Error },
            ResponseBytesError { source: reqwest::Error },
            DeserializeError { source: serde_json::Error, body: bytes::Bytes },
        }
    }

snafu creates structs in the module for each one of the enum values. You reference the structs, from the operation in a .context() or creating it directly and calling fail() when it isn't wrapping a source.

Feedback welcome. I imagine there may be benefits to swapping out thiserror for snafu in our sdk, but they both work with std::error::Error.

TokenCredential is currently in azure_identity. We discussed moving it to azure_core I'd like to reference it from the generated services, but it should be moved first.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/get-queue-service-properties.

When uploading a blob I can successfully upload a metadata hashmap, now I need to be able to fetch that without downloading the actual (big) blob. For this azure has the get-blob-metadata call (https://docs.microsoft.com/en-us/rest/api/storageservices/get-blob-metadata).

Can someone point me to how one approaches adding such a missing api?

I can't seem to find a sensible way to page storage table query results.

There isn't an obvious way (to me) to create a Continuation at a given point, nor does there appear to be a good way to serialize a Continuation. I would like to provide a simple RESTful API that offers a single string next that can be given to the API again to continue a query. This SDK doesn't appear to let me do that.

I'm not all that good at rust yet, but I think what I want is for Continuation to implement the traits ToString and FromStr, so that I can simply return a string to the API caller that they can pass back to me for the next page.

Currently the core crate is a bit of a mess with many seemingly unrelated types jammed together into one namespace. Many of these types can be moved into the newly consolidated storage crate.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/get-queue-metadata.

Currently PartitionKeys and stored_procedure::Parameters are simple aliases of ToJsonVector. I'm not convinced that this wins us anything over simply using a wrapper around Vec<String> for each of these (or simply Vec<String> itself in the case of Parameters). Switching to a simpler representation would most likely make these types easier to understand as well.

Support Copy Blob:
https://docs.microsoft.com/en-us/rest/api/storageservices/Copy-Blob

This requires other dependencies to upgrade first:

• hyper v0.14 hyperium/hyper#2302
• hyper v0.14 https://github.com/hyperium/hyper/milestone/6
• reqwest ? seanmonstar/reqwest#1060

As per https://docs.microsoft.com/en-us/rest/api/storageservices/set-queue-service-properties.

rust Command cannot find az command on (some) windows systems. Most certainly it is related to how command is implemented:
see rust-lang/rust#42791 and https://doc.rust-lang.org/nightly/std/process/struct.Command.html.

I think az is az.cmd on windows and it should be called with cmd /c

let az_command = Command::new("cmd")
            .args(&[
                "/C",
                "az",
                "account",
                "get-access-token",
                "--output",
                "json",
                "--resource",
                resource,
            ])
            .output();

Hi! Thanks for all the hard work on this SDK so far, it's been great to work with.

It looks like the Get Blob Properties method is missing from the Blob trait. Is there a reason it's missing, or would you be willing to accept a PR to add it?

As per https://docs.microsoft.com/en-us/rest/api/storageservices/peek-messages.

try next with wrong az account:

    let mut key_vault_client = KeyVaultClient::new(&creds, "lahodapro");
    println!("list secrets for");
    let secrets = key_vault_client.list_secrets().await?;

Expected: error message about wrong auth usage
Actual: error about some value is missing
Notes: that is very important that entry point to azure via auth to have guiding errors, not cryptic

As per https://docs.microsoft.com/en-us/rest/api/storageservices/delete-queue3.

As per https://docs.microsoft.com/en-us/rest/api/storageservices/get-queue-service-stats.

Add a default User-Agent header. This will allow telemetry to be collected and help build a case for making this SDK more official.

User-Agent: <product> / <product-version> <comment>
User-Agent: Mozilla/5.0 (<system-information>) <platform> (<platform-details>) <extensions>
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

.NET

https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/core/Azure.Core/src/Pipeline/Internal/TelemetryPolicy.cs
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/avs/Microsoft.Azure.Management.Avs/tests/SessionRecords/AvsTests/AvsCrud.json#L15-L20

        "User-Agent": [
          "FxVersion/4.6.28928.01",
          "OSName/Windows",
          "OSVersion/Microsoft.Windows.10.0.19041.",
          "Microsoft.Azure.Management.ResourceManager.ResourceManagementClient/1.6.0.0"
        ],

Python

From the az cli:

'User-Agent': 'python/3.6.6 (Windows-10-10.0.19041-SP0) msrest/0.6.13 msrest_azure/0.6.3 avsclient/2020-03-20 Azure-SDK-For-Python AZURECLI/2.5.0 (MSI)'