Giter VIP home page Giter VIP logo

azureml-sdk-for-r's Introduction

Azure Machine Learning SDK for R (preview)

Build Status Build Status Build Status Build Status CRAN_Status_Badge

** The Azure Machine Learning SDK for R was deprecated at the end of 2021 to make way for an improved R training and deployment experience using Azure Machine Learning CLI 2.0. See the samples repository to get started with the 2.0 CLI. **

Data scientists and AI developers use the Azure Machine Learning SDK for R to build and run machine learning workflows with Azure Machine Learning.

Azure Machine Learning SDK for R uses the reticulate package to bind to Azure Machine Learning's Python SDK. By binding directly to Python, the Azure Machine Learning SDK for R allows you access to core objects and methods implemented in the Python SDK from any R environment you choose.

Main capabilities of the SDK include:

  • Manage cloud resources for monitoring, logging, and organizing your machine learning experiments.
  • Train models using cloud resources, including GPU-accelerated model training.
  • Deploy your models as webservices on Azure Container Instances (ACI) and Azure Kubernetes Service (AKS).

Please take a look at the package website https://azure.github.io/azureml-sdk-for-r for complete documentation.

Key Features and Roadmap

โœ”๏ธ feature available ๐Ÿ”„ in progress ๐Ÿ“‹ planned

Features Description Status
Workspace The Workspace class is a foundational resource in the cloud that you use to experiment, train, and deploy machine learning models โœ”๏ธ
Compute Cloud resources where you can train your machine learning models. โœ”๏ธ
Data Plane Resources Datastore, which stores connection information to an Azure storage service, and DataReference, which describes how and where data should be made available in a run. โœ”๏ธ
Experiment A foundational cloud resource that represents a collection of trials (individual model runs). โœ”๏ธ
Run A Run object represents a single trial of an experiment, and is the object that you use to monitor the asynchronous execution of a trial, store the output of the trial, analyze results, and access generated artifacts. You use Run inside your experimentation code to log metrics and artifacts to the Run History service. โœ”๏ธ
Estimator A generic estimator to train data using any supplied training script. โœ”๏ธ
HyperDrive HyperDrive automates the process of running hyperparameter sweeps for an Experiment. โœ”๏ธ
Model Cloud representations of machine learning models that help you transfer models between local development environments and the Workspace object in the cloud. โœ”๏ธ
Webservice Models can be packaged into container images that include the runtime environment and dependencies. Models must be built into an image before you deploy them as a web service. Webservice is the abstract parent class for creating and deploying web services for your models. โœ”๏ธ
Dataset An Azure Machine Learning Dataset allows you to explore, transform, and manage your data for various scenarios such as model training and pipeline creation. When you are ready to use the data for training, you can save the Dataset to your Azure ML workspace to get versioning and reproducibility capabilities. TabularDataset support is experimental. โœ”๏ธ

Installation

Install Conda if not already installed. Choose Python 3.5 or later.

# Install Azure ML SDK from CRAN
install.packages("azuremlsdk")

# Or the development version from GitHub
install.packages("remotes")
remotes::install_github('https://github.com/Azure/azureml-sdk-for-r', build_vignettes = TRUE)

# Then, use `install_azureml()` to install the compiled code from the AzureML Python SDK.
azuremlsdk::install_azureml()

Now, you're ready to get started!

For a more detailed walk-through of the installation process, advanced options, and troubleshooting, see our Installation Guide.

Getting Started

To begin running experiments with Azure Machine Learning, you must establish a connection to your Azure Machine Learning workspace.

  1. If you don't already have a workspace created, you can create one by doing:

    # If you haven't already set up a resource group, set `create_resource_group = TRUE`  
    # and set `resource_group` to your desired resource group name in order to create the resource group 
    # in the same step.
    new_ws <- create_workspace(name = <workspace_name>, 
                               subscription_id = <subscription_id>, 
    			   resource_group = <resource_group_name>, 
    			   location = location, 
    			   create_resource_group = FALSE)

    After the workspace is created, you can save it to a configuration file to the local machine.

    write_workspace_config(new_ws)
  2. If you have an existing workspace associated with your subscription, you can retrieve it from the server by doing:

    existing_ws <- get_workspace(name = <workspace_name>, 
    			     subscription_id = <subscription_id>, 
    			     resource_group = <resource_group_name>)

    Or, if you have the workspace config.json file on your local machine, you can load the workspace by doing:

    loaded_ws <- load_workspace_from_config()

Once you've accessed your workspace, you can begin running and tracking your own experiments with Azure Machine Learning SDK for R.

Take a look at our code samples and end-to-end vignettes for examples of what's possible with the SDK!

Resources

Contribute

We welcome contributions from the community. If you would like to contribute to the repository, please refer to the contribution guide.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azureml-sdk-for-r's People

Contributors

abeomor avatar danielsc avatar diondrapeck avatar heemanshusuri avatar mx-iao avatar ninghu avatar revodavid avatar sdgilley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azureml-sdk-for-r's Issues

load_workspace_from_config() is unable to load workspace when using LocalWebservice

Describe the bug
load_workspace_from_config() is unable to load workspace when using LocalWebservice

To Reproduce

ws = Workspace.from_config()
model = Model(ws, 'model')

environment = Environment("LocalDeploy")
environment.python.conda_dependencies = CondaDependencies("env/conda_dependencies.yml")

inference_config = InferenceConfig(entry_script="score.py", environment=environment)
deployment_config = LocalWebservice.deploy_configuration(port=8890)
service = Model.deploy(
    ws, "test", [model], inference_config, deployment_config)

Expected behavior
Should be able to load without errors.

Screenshots
image

Additional context
Docker version 18.05.0-ce, build f150324
azureml-core version: 1.0.85.2

Deployment times out: train-and-deploy-aci vignette

I had come to an issue trying to run the chunk on deployment:

aci_service <- deploy_model(ws, 
                            'accident-pred2', 
                            list(model), 
                            inference_config, 
                            aci_config)

wait_for_deployment(aci_service, show_output = TRUE)

## Running............................................................................................................................
## TimedOut
## ERROR - Service deployment polling reached non-successful terminal state, current service state: ## Unhealthy
## Operation ID: e40a93cc-680d-4aa0-a975-2964d2e92a5d
## More information can be found using '.get_logs()'
## Error:
## {
##  "code": "DeploymentTimedOut",
##  "statusCode": 504,
##  "message": "The deployment operation polling has TimedOut. The service creation is taking longer than our normal time. We are still trying to achieve the desired state for the web service. 
## Please check the webservice state for the current webservice health. You can run 
## print(service.state) from the python SDK to retrieve the current state of the webservice."
## }

## ERROR - Service deployment polling reached non-successful terminal state, current service state: ## Unhealthy
## Operation ID: e40a93cc-680d-4aa0-a975-2964d2e92a5d
## More information can be found using '.get_logs()'
## Error:
## {
##  "code": "DeploymentTimedOut",
##  "statusCode": 504,
##   "message": "The deployment operation polling has TimedOut. The service creation is taking longer than our normal time. We are still trying to achieve the desired state for the web service. 
## Please check the webservice state for the current webservice health. You can run 
## print(service.state) from the python SDK to retrieve the current state of the webservice."
## }

## Error in py_call_impl(callable, dots$args, dots$keywords) : 
##  WebserviceException: WebserviceException:
## 	Message: Service deployment polling reached non-successful terminal state, current service state: Unhealthy
## Operation ID: e40a93cc-680d-4aa0-a975-2964d2e92a5d
## More information can be found using '.get_logs()'
## Error:
## {
##  "code": "DeploymentTimedOut",
##  "statusCode": 504,
##  "message": "The deployment operation polling has TimedOut. The service creation is taking longer than our normal time. We are still trying to achieve the desired state for the web service. 
## Please check the webservice state for the current webservice health. You can run 
## print(service.state) from the python SDK to retrieve the current state of the webservice."
## }
##	InnerException None
##	ErrorResponse 
## {
##    "error": {
##        "message": "Service deployment polling reached non-successful terminal state, current 
## service state: Unhealthy\nOperation ID: e40a93cc-680d-4aa0-a975-2964d2e92a5d\nMore 
## information can be found using '.get_logs()'\nError:\n{\n  \"code\": \"Deploymen

Using reticulate, I have been trying to access the .get_logs() from the service object without much success. Is there anyways to access it to find more details on the error? Is this issue ocurring only on my Azure session?

The details of the R session are as follows:

## R version 3.6.2 (2019-12-12)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 16.04.6 LTS

## Matrix products: default
## BLAS:   /usr/lib/libblas/libblas.so.3.6.0
## LAPACK: /home/azureuser/.local/share/r-miniconda/envs/r-reticulate/lib/libmkl_rt.so

## locale:
## [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
## [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     

## other attached packages:
## [1] azuremlsdk_0.6.85

## loaded via a namespace (and not attached):
## [1] compiler_3.6.2  tools_3.6.2     rappdirs_0.3.1  Rcpp_1.0.3      reticulate_1.14 knitr_1.26      jsonlite_1.6    xfun_0.11  

view_run_details function does not work in RStudio Server on Compute Instance

Describe the bug
view_run_details() widget does not work within RStudio Server session on new Compute Instances which it does work in RStudio Server on Notebook VMs. The view_run_details code incorrectly extracts the instance name and domain_suffix from "~/../../mnt/azmnt/.nbvm" so defaults to localhost as the widget endpoint in the viewer/browser. On both NBVM and CI, the file exists and is in the same location but on NBVM, the file contains a blank line at line 1. Instance_name and domain_suffix are read from lines 2 and 3. On CI, the blank line is no longer there so instance name and domain_suffix are read incorrectly.

To Reproduce

  1. Create an instance of Compute Instance in a region where CI is available
  2. From RStudio Server on the CI, run the train local sample for the R SDK.
  3. RStudio Viewer tries to open run status widget at localhost and is unable to dsplay.

Expected behavior
Viewer display run details widget:
image

Screenshots
In Jobs tab: [1] "Listening on 127.0.0.1:7580".
In RStudio Viewer when failing: image

Additional context
Fix: if length(nvbm) after the readLines == 3, the file is on a NVBM -> read lines 2 and 3. If length(nvbm) == 2 ,the file is on a CI -> read lines 1 and 2.
Temp workaround on CI: edit "~/../../mnt/azmnt/.nbvm" to add blank line at top of file.

train-with-tensorflow: deprecation warnings

Describe the bug
When running the vignette "train-with-tensorflow" the user sees several warning messages about deprecated functions being used

To Reproduce
From tf_mnist.R:

> W <- tf$Variable(tf$zeros(shape(784L, 10L)))
WARNING:tensorflow:From /anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
> train_step <- tf$train$GradientDescentOptimizer(0.5)$minimize(cross_entropy)
WARNING:tensorflow:From /anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
> mnist <- datasets$mnist$read_data_sets("MNIST-data", one_hot = TRUE)
WARNING:tensorflow:From /anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:320: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.

Expected behavior
Tutorial uses non-deprecated functions instead.

Additional context
This is using RStudio Server in an AML Compute Instance. R version 3.6.2.

Error: Python module azureml was not found.

Hello, here is installation as per the docs:

library(azuremlsdk)

azuremlsdk::install_azureml(version = "1.0.69", envname = "r-azureml", conda_python_version = "3.6", restart_session = TRUE, remove_existing_env = TRUE)

It results in:

> get_current_run() Error: Python module azureml was not found.

Sessioninfo:
R version 3.5.0 (2018-04-23)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] azuremlsdk_0.5.7 sendmailR_1.2-1 doSNOW_1.0.18 snow_0.4-3 iterators_1.0.12 foreach_1.4.7 assertthat_0.2.1 RMySQL_0.10.18
[9] DBI_1.1.0 stringr_1.4.0 plyr_1.8.5 zoo_1.8-7 lubridate_1.7.4 XML_3.99-0.3 xml2_1.2.2 jsonlite_1.6
[17] tidyr_1.0.2 dplyr_0.8.3 magrittr_1.5

loaded via a namespace (and not attached):
[1] Rcpp_1.0.3 pillar_1.4.3 compiler_3.5.0 base64enc_0.1-3 tools_3.5.0 lifecycle_0.1.0 tibble_2.1.3 lattice_0.20-35
[9] pkgconfig_2.0.3 rlang_0.4.4 rstudioapi_0.10 parallel_3.5.0 vctrs_0.2.2 grid_3.5.0 tidyselect_1.0.0 reticulate_1.14
[17] glue_1.3.1 R6_2.4.1 purrr_0.3.3 codetools_0.2-15 stringi_1.4.5 crayon_1.3.4

Error while installing azureml: "Your current architecture is 32bit"

Describe the bug
While following the instructions in README.md to install the azureml package, the install fails with the error Your current architecture is 32bit however this version of Python is compiled for 64bit.

To Reproduce
Install azureml with:

devtools::install_github('https://github.com/Azure/azureml-sdk-for-r', auth_token = '<personal access toke>')

This is the output I get. I am using a 64-bit version of R from RStudio, and the 64-bit Python Anaconda version.

Downloading GitHub repo Azure/azureml-sdk-for-r@master
โˆš  checking for file 'C:\Users\davidsmi\AppData\Local\Temp\RtmpsXzBDj\remotes35ec72d8325c\Azure-azureml-sdk-for-r-5d76e261c076ad4dd92ededb6ea67f41b2108577/DESCRIPTION' (913ms)
-  preparing 'azureml': (702ms)
โˆš  checking DESCRIPTION meta-information ... 
-  checking for LF line-endings in source and make files and shell scripts (448ms)
-  checking for empty or unneeded directories
-  building 'azureml_0.1.0.tar.gz'
   
Installing package into โ€˜C:/Users/davidsmi/Documents/R/win-library/3.6โ€™
(as โ€˜libโ€™ is unspecified)
* installing *source* package 'azureml' ...
** using staged installation
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package 'azureml'
    finding HTML links ... done
    azureml                                 html  
    create_aml_compute                      html  
    create_conda_env                        html  
    create_estimator                        html  
    create_launch_script                    html  
    create_run_config                       html  
    create_script_run_config                html  
    create_workspace                        html  
    delete_aml_compute                      html  
    delete_workspace                        html  
    download_from_datastore                 html  
    experiment                              html  
    get_compute                             html  
    get_current_run                         html  
    get_default_datastore                   html  
    get_run_metrics                         html  
    get_workspace                           html  
    install_azureml                         html  
    list_workspaces                         html  
    load_workspace_from_config              html  
    log_metric_to_run                       html  
    submit_experiment                       html  
    upload_files_to_datastore               html  
    upload_to_datastore                     html  
    wait_for_compute                        html  
    wait_for_run_completion                 html  
    write_workspace_config                  html  
** building package indices
** testing if installed package can be loaded from temporary location
*** arch - i386
Error: package or namespace load failed for 'azureml':
 .onLoad failed in loadNamespace() for 'azureml', details:
  call: NULL
  error: Your current architecture is 32bit however this version of Python is compiled for 64bit.
Error: loading failed
Execution halted
*** arch - x64
Error: package or namespace load failed for 'azureml':
 .onLoad failed in loadNamespace() for 'azureml', details:
  call: NULL
  error: ModuleNotFoundError: No module named 'azureml'
Error: loading failed
Execution halted
ERROR: loading failed for 'i386', 'x64'
* removing 'C:/Users/davidsmi/Documents/R/win-library/3.6/azureml'
Error: Failed to install 'azureml' from GitHub:
  (converted from warning) installation of package โ€˜C:/Users/davidsmi/AppData/Local/Temp/RtmpsXzBDj/file35ec7d6e7f98/azureml_0.1.0.tar.gzโ€™ had non-zero exit status

Expected behavior
The azureml package installs without error.

load_workspace_from_config() is unable to load workspace from notebook VM

Describe the bug
load_workspace_from_config() is unable to load workspace from notebook VM

To Reproduce

ws <- load_workspace_from_config()
Error in py_call_impl(callable, dots$args, dots$keywords) :
UserErrorException: UserErrorException:
Message: We could not find config.json in: /home/azureuser/azureml-sdk-for-r/samples/training/train-on-amlcompute or in its parent directories. Please provide the full path to the config file or ensure that config.json exists in the parent directories.
InnerException None
ErrorResponse
{
"error": {
"code": "UserError",
"message": "We could not find config.json in: /home/azureuser/azureml-sdk-for-r/samples/training/train-on-amlcompute or in its parent directories. Please provide the full path to the config file or ensure that config.json exists in the parent directories."
}

Expected behavior
The workspace should be able to be loaded without errors.

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
Add any other context about the problem here.

add dataset_consumption_config() method

Add dataset_consumption_config <- function(name, dataset, mode = 'direct', path_on_compute = NULL) method so that a user can pass a list of these into the inputs parameter of the estimator() method without having to do inputs = list(dataset$as_named_input('mydataset')$as_download()), for instance.

New code would be: inputs = list(dataset_consumption_config('mydataset', dataset, mode = 'download'))

view_run_details(run): Error in loadNamespace(name) : there is no package called โ€˜knitrโ€™

Describe the bug
Got below error when call view_run_details(run) in vignette on a new notebook VM.

Error in loadNamespace(name) : there is no package called โ€˜knitrโ€™

To Reproduce

  1. Open RStudio from a new notebook VM
  2. Run through train-with-tensorflow vignette
  3. Get an error when call view_run_details()
    Error in loadNamespace(name) : there is no package called โ€˜knitrโ€™

Expected behavior
No Error

Screenshots
If applicable, add screenshots to help explain your problem.
image

Warnings are shown in the console after view_run_details() is called

Describe the bug
Warnings are shown in the console after view_run_details() is called

To Reproduce

  1. Submit a run
  2. Call view_run_details(run)

Expected behavior
The widget should be shown in the viewer pane without any warnings in the console

Screenshots
image

Additional context
N/A

view_run_details(run) throw exception Error in parse_iso_8601(details$endTimeUtc)

Describe the bug
view_run_details(run) throw exception Error in parse_iso_8601(details$endTimeUtc)

To Reproduce
After run is completed, run )

view_run_details(run)
Error in parse_iso_8601(details$endTimeUtc) :
could not find function "parse_iso_8601"

Expected behavior
The widget should be refreshed in Viewer pane without error.

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
Add any other context about the problem here.

Support for Pipelines?

I'm looking for pipeline support similar to the Python version. I have 2 customers doing databricks w/ R who would like to use pipelines over top of these steps with datasets etc.

I do not see pipelines on the readme as a supported or planned to support item. Is it supported? If not, can we please ensure it is on the map?

Need ability to support pipeline creations from CLI & .yaml variations for multi step R projects supporting databricks and AML compute.

add data_path() so that create dataset methods can take in DataPaths

create_file_dataset_from_files() and create_tabular_dataset_from_*() methods can take in DataPath or list of DataPaths to the path parameter to specify datastore + path.

In order to support that we will need to add a data_path <- function(datastore = NULL, path_on_datastore = NULL, name = NULL) method (see Python).

The create methods' path parameter can also take in a tuple of (datastore, path) but I could not get that to work when using the R SDK with list(datastore, path). Let me know if I'm doing something wrong. Either way I think it'll be cleaner to support & document taking in DataPaths.

AttributeError: module 'azureml' has no attribute 'core'

Describe the bug
I'm having issues with Azure Machine Learning SDK for R: "module 'azureml' has no attribute 'core'"...

I am trying to use azureml to apply machine learning (my own stuff, written in R) to data from our data warehouse that is put in the blob storage. The modelled output should be put back into the blob storage so it can be accessed from the data warehouse. I've written the code in R on my local machine (stored in a git repo). Preferably, I'd find some method to pull my code from git into a pipeline in the azureml environment so that it can be directly run whenever new data is available in the blob storage.

I've embarked on a tutorial-spree and found this seemingly relevant walkthrough: [Train and deploy your first model with Azure ML] (https://azure.github.io/azureml-sdk-for-r/#getting-started).

But... after trying all I could think of, I'm stuck on the first steps. After installing all (or at least.. that's what I think) packages, modules, apps etc, and running the following code in RStudio:

To Reproduce
library(azuremlsdk) existing_ws <- get_workspace(name = name, subscription_id = subscription_id, resource_group)

I run into an error that I haven't been able to fix:

AttributeError: module 'azureml' has no attribute 'core'

Expected behavior
I expected to be able to load the workspace.

Additional context
It seems that the azuerml is supposed to have an attribute "core", but when looking at it more closely, there is indeed no such attribute. The function "get_workspace()" is trying to access: "azureml$core$Workspace$get". I found that "azuerML$Workspace" does exist, but then I can't figure out how to make that work.

Can anyone explain to me why I'm encountering this error? Does anyone know of a better tutorial on how to connect my R code the azureml's cloud service?

Any pointers in the right direction are much appreciated!

I also tried stackoverflow, but to no avail.

Apologies if this is not posted in the correct location, but I've been stuck at this for a while now. Any help towards a solution would be much appreciated.

AttributeError: 'AciWebservice' object has no attribute 'reload'

Describe the bug
reload_local_webservice_assets(service) throws below error

AttributeError: 'AciWebservice' object has no attribute 'reload'

To Reproduce
Steps to reproduce the behavior:

Run sample training\train-on-local

Expected behavior
reload_local_webservice_assets(service) call should fail.

Screenshots
image

vignettes not discoverable

Describe the bug
Most vignettes are not listed at https://cran.r-project.org/package=azuremlsdk ; only the installation and configuration vignettes are shown. Same goes for vignette(package="azuremlsdk") and the "vignettes" link shown by help(package="azuremlsdk")

To Reproduce

vignette(package="azuremlsdk")

Expected behavior
All vignettes are listed.

Screenshots
On CRAN:
image

In Help window after help(package="azuremlsdk"):
image

Additional context
I think this is due to most vignettes being provded in subfolders of the vignettes folder.

Driver log would be more useful with commands

Is your feature request related to a problem? Please describe.
When an error occurs in a training script, the first place to look is the driver log file. But this file only includes R outputs. If it included the command (inputs) as well, it would be more useful. Otherwise, if there is a long block of commands that generate no output, it's hard to tell which command generated the error.

Describe the solution you'd like
If you are using source to process the train.R file, you could use:

source("train.R", echo=TRUE, prompt.echo="> ")

Workspace related functions on Databricks

Describe the bug
I am trying to use this library in Databricks and I encounter a problem in the operation of this function (it hangs).

To Reproduce

  • install the azureml sdk in this way so that databricks work on this enviroment
    azuremlsdk::install_azureml(envname = "databricks-standard")

  • Then I run 'get_current_run()' to check that it went well, and return:
    <azureml.core.run._OfflineRun>

  • None of the Azure Workspace related commands work

Installation steps are inconsistent

Describe the bug
The Github installation readme works fine but it is diffrent from the installation.Rmd file.
The installation.Rmd is missing the following step.

Then, use install_azureml() to install the compiled code from the AzureML Python SDK.
> azureml::install_azureml() 
Now, you're ready to get started!

The README.md is pointing to the wrong link when it directs you to more instructions

Here For a more detailed walk-through of the installation process, advanced options, and troubleshooting, see our Installation Guide.
https://github.com/Azure/azureml-sdk-for-r/blob/master/docs/articles/installation.html

provide default value for source_directory param in inference_config()

We should provide a default value source_directory = "." for the inference_config() method, since if source_directory = NULL only the file provided to entry_script is uploaded by Azure ML. Since that file is _generated_score.py, the user-provided score.R file won't get picked up and deployment will fail.

optparse not included in default image

Describe the bug
When an experiment is processed, we can see that the getopt package is being installed. Since every training script requires this package, this unneccessarily slows down training.

To Reproduce

run <- submit_experiment(exp, est)
view_run_details(run)
wait_for_run_completion(run, show_output = TRUE)

The output includes:

> install.packages('optparse', repos = 'http://cran.us.r-project.org')
also installing the dependency โ€˜getoptโ€™

Expected behavior
The getopt package should already be installed in the image so that this step can be skipped.

package e1071 is required

Describe the bug
Training script fails with the error:

Error: package e1071 is required
Execution halted

As far as I can tell I am not using any function from e1071, and so it was not included in the cran_packages option to estimator. Adding "e1071" to cran_packages works around the issue.

To Reproduce
https://github.com/revodavid/azure-ml-R-tutorial/tree/77f1f5b34d033764c410ef78921a1d46767a843f

Expected behavior
If e1071 is always required, it should be loaded by default or documented as such. It's an odd dependency to have.

Unable to install the azureml sdk for R

Describe the bug
When I try installing the package using :
remotes::install_github('https://github.com/Azure/azureml-sdk-for-r', auth_token = 'rppo2tmxykypkkyjklrzbzyzwtnuwzpvpekafh7my5lf65ynydva',INSTALL_opts=c("--no-multiarch"), build_vignettes = FALSE)

I get this
Error: Failed to install 'unknown package' from GitHub: cannot open URL 'http://api.github.com/repos/Azure/azureml-sdk-for-r/contents/DESCRIPTION?ref=master'

To Reproduce
Create a new conda environment
install the R environment from R
install the remotes package
run this command :
remotes::install_github('https://github.com/Azure/azureml-sdk-for-r', auth_token = 'rppo2tmxykypkkyjklrzbzyzwtnuwzpvpekafh7my5lf65ynydva',INSTALL_opts=c("--no-multiarch"), build_vignettes = FALSE)

Expected behavior
I expect this to work.

Additional context
I build a new conda environment with Python 3.6, and installed the R distribution from Conda (the R channel)

train-and-deploy-to-aci vignette deployment times out

Describe the bug
Get the below error when trying to deploy web service to ACI

To reproduce

  • Run the train-and-deploy-to-aci vignette
  • Try deploying the web service

Expected behavior
Service deploys successfully and aci_service$state == "Healthy"

Screenshots
serviceerror

Logs from running get_webservice_logs(aci_service):

get_webservice_logs(aci_service)
[1] "2019-10-25T17:43:04,864103860+00:00 - rsyslog/run \n2019-10-25T17:43:04,865977822+00:00 - iot-server/run \n2019-10-25T17:43:04,872023520+00:00 - gunicorn/run \n2019-10-25T17:43:04,873996184+00:00 - nginx/run \nEdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting...\n2019-10-25T17:43:05,174359213+00:00 - iot-server/finish 1 0\n2019-10-25T17:43:05,175580253+00:00 - Exit code 1 is normal. Not restarting iot-server.\nStarting gunicorn 19.9.0\nListening at: http://127.0.0.1:31311 (15)\nUsing worker: sync\nworker timeout is set to 300\nBooting worker with pid: 47\nInitialized PySpark session.\nInitializing logger\nStarting up app insights client\nStarting up request id generator\nStarting up app insight hooks\nInvoking user's init function\nUser's init function failed\nEncountered Exception Traceback (most recent call last):\n File "/var/azureml-server/aml_blueprint.py", line 162, in register\n main.init()\n File "/var/azureml-app/main.py", line 88, in init\n driver_module.init()\n File "_generated_score.py", line 15, in init\n robjects.r.source("{}".format(score_r_path))\n File "/opt/miniconda/lib/python3.7/site-packages/rpy2/robjects/functions.py", line 178, in call\n return super(SignatureTranslatedFunction, self).call(*args, **kwargs)\n File "/opt/miniconda/lib/python3.7/site-packages/rpy2/robjects/functions.py", line 106, in call\n res = super(Function, self).call(*new_args, **new_kwargs)\nrpy2.rinterface.RRuntimeError: Error in file(filename, "r", encoding = encoding) : \n cannot open the connection\n\n\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: Error in file(filename, "r", encoding = encoding) : \n cannot open the connection\n\n warnings.warn(x, RRuntimeWarning)\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: In addition: \n warnings.warn(x, RRuntimeWarning)\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: Warning message:\n\n warnings.warn(x, RRuntimeWarning)\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: In file(filename, "r", encoding = encoding) :\n warnings.warn(x, RRuntimeWarning)\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: \n \n warnings.warn(x, RRuntimeWarning)\n/opt/miniconda/lib/python3.7/site-packages/rpy2/rinterface/init.py:146: RRuntimeWarning: cannot open file '/var/azureml-app/accident_predict.R': No such file or directory\n\n warnings.warn(x, RRuntimeWarning)\nWorker exiting (pid: 47)\nShutting down: Master\nReason: Worker failed to boot.\n2019-10-25T17:43:06,628225084+00:00 - gunicorn/finish 3 0\n2019-10-25T17:43:06,629504826+00:00 - Exit code 3 is not normal. Killing image.\n"

Error is missing from the widget

Describe the bug
Error is missing from the widget for a failed run.

To Reproduce

  1. Submit a job which would fail in train script.
  2. Call show_run_details(run)

Expected behavior
Error is supposed to be shown at the bottom of the table but it wasn't

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
Add any other context about the problem here.

Add "See Also" links to help pages

None of the function help pages in the azuremlsdk library include a See Also section. Including relevant functions in a See Also section would help users discover functions and navigate the help system better.

Describe the solution you'd like
Add a see also section to the help pages listing relevant functions. For example, upload_files_to_datastore should refer to download_from_datastore and vice versa.

Additional context
A "See Also" section is standard for R help files, and the R system makes referenced functions clickable links in help pages. You just need to provide function names, not URLs or file paths.

azureml.get_model_path dont search in azureml-models

I'm trying to load a model from azureml that I've registred previouly. I have checked that the model is save, but when I use azureml$get_model_path("model_name") return an error Model not found in cache or in root. Why this function dont search in azureml-models?

azureml SDK for Python is a prerequesite

Describe the bug
I haven't verified this, but I assume the azureml Python SDK is a prerequsite.

To Reproduce
Try installing and using this package without the Python SDK installed.

Expected behavior
The README.md file should document this prerequisite.

Log plots with Log_Image

Describe the solution you'd like

The Python SDK supports logging plots in a run with

run.log_image(name, path=None, plot=None)

and can take a matplotlib plot as an argument. It would be nice if the R SDK can take plots as arguments (especially ggplot2 plots).

grid_parameter_sampling with choice errors with fixed values

Hi,
I'm trying to get a hyperparameter example working using grid_parameter_sampling. When using vectors to supply discrete values such as "nrounds" = choice(c(100,200)) this works fine. If however I wish to fix a parameter I am unable to. For example "gamma"= choice(c(0)) returns error:

Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: 'float' object is not iterable

As does "gamma"= choice(0). "gamma"= 0 returns:

Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: 'float' object is not subscriptable

The documentation for choice states that it will accept either an integer vector or one or more comma separated discrete values. The first condition seems to be true but if you supply multiple discreet values separated by commas the following error is returned:

Error in choice(1, 2) : unused argument (2)

I'm guessing the last case is due to R documentation being aligned to the python doco.

For the grid_parameter_sampling function to be useful it needs to be able to fix parameters and allow a list of discrete values of types other than integer (i.e. float, char)

Regards,

Ashley

Add function to register model to run

Is your feature request related to a problem? Please describe.
There isn't a wrapped function to allow registering a model with a run - only for registering a model to a workspace. Both are possible in Python SDK, and we should have parity. This is a user request.

Describe the solution you'd like
Add register_model_to_run function

Strange named component "datatables htmlwidget" in run viewer

Describe the bug
Using view_run_details within an R Markdown chunk creates two "tabs", "datatables htmlwidget" and "R Console". The second name makes sense but the first is confusing.

To Reproduce
See screenshot

Expected behavior
"Azure ML Run Log" might be a better name.

Screenshots
image

Additional context
It's possible the "datatables htmlwidget" name is created by RStudio itself.

widget not showing up in Viewer pane in RStudio when running from RMarkdown file

Describe the bug
The widget is not showing up in Viewer, only in the code chunk output of the Rmd file.

To Reproduce
Run view_run_details(run), for example from the train-and-deploy-to-aci vignette.

Expected behavior
Widget to show up in RStudio Viewer.

Screenshots
rstudio-widget

This error shows up in console after view_run_details() is called:
widgeterror

pip install azureml-sdk failing: Could not find a version that satisfies the requirement dotnetcore2>=2.1.9

Describe the bug
pip install azureml-sdk failing on Windows 10 Enterprise Insider Preview Dell XPS 15 laptop (build 19551.1001) with Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]

To Reproduce
pip install azureml-sdk

Expected behavior
I expect pip to install the AzureML SDK

Actual behavior

ERROR: Could not find a version that satisfies the requirement dotnetcore2>=2.1.9 (from azureml-dataprep[fuse]<1.2.0a,>=1.1.37a->azureml-sdk) (from versions: none)
ERROR: No matching distribution found for dotnetcore2>=2.1.9 (from azureml-dataprep[fuse]<1.2.0a,>=1.1.37a->azureml-sdk)

Screenshots
Screenshot 2020-01-24 09 17 00

Additional context
I have downloaded and installed the .Net Core SDK and runtime from https://dotnet.microsoft.com/download/

deploy-to-local sample get permission denied error when run on notebook vm

Describe the bug
deploy-to-local sample get permission denied error when run on notebook vm

To Reproduce
service <- deploy_model(ws,
'rservice-local',
list(model),
inference_config,
local_deployment_config)
ERROR - Failed to create Docker client. Is Docker running/installed?
When you deploy locally, we download a dockerfile
execute docker build on it, and docker run the built container for you
Error: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))

Error in py_call_impl(callable, dots$args, dots$keywords) :
WebserviceException: WebserviceException:
Message: Failed to create Docker client. Is Docker running/installed?
When you deploy locally, we download a dockerfile
execute docker build on it, and docker run the built container for you
Error: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
InnerException None
ErrorResponse
{
"error": {
"message": "Failed to create Docker client. Is Docker running/installed?\nWhen you deploy locally, we download a dockerfile\nexecute docker build on it, and docker run the built container for you\nError: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"
}
}

Expected behavior
Shouldn't get any error for local deployment.

Error in py_call_impl

Describe the bug
When I try creating a workspace, I got the following error.

> new_ws <- create_workspace(name = ...
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  TypeError: create() got an unexpected keyword argument 'sku'

To Reproduce
I have just installed everything.

Expected behavior
I can create a workspace.

Additional context
My environment is:

Tool Version
OS Windows 10 Pro x64 10.0.18362
R 3.6.1 x64
Python 3.7 with Anaconda

r-azureml environment seems successfully installed as I can see it in Anaconda Navigator and see the following log.

> library(azuremlsdk)
> azuremlsdk::install_azureml()
Creating environment:  r-azureml
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\kemakino\AppData\Local\CONTIN~1\ANACON~1\envs\r-azureml

  added / updated specs:
    - python=3.6


The following NEW packages will be INSTALLED:

  certifi            pkgs/main/win-64::certifi-2019.11.28-py36_0
  pip                pkgs/main/win-64::pip-19.3.1-py36_0
  python             pkgs/main/win-64::python-3.6.9-h5500b2f_0
  setuptools         pkgs/main/win-64::setuptools-42.0.2-py36_0
  sqlite             pkgs/main/win-64::sqlite-3.30.1-he774522_0
  vc                 pkgs/main/win-64::vc-14.1-h0510ff6_4
  vs2015_runtime     pkgs/main/win-64::vs2015_runtime-14.16.27012-hf0eaf9b_0
  wheel              pkgs/main/win-64::wheel-0.33.6-py36_0
  wincertstore       pkgs/main/win-64::wincertstore-0.2-py36h7fe50ca_0


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate r-azureml
#
# To deactivate an active environment, use
#
#     $ conda deactivate


C:\Users\kemakino\src\kemakino\ttp289a-2019\ttp289a-2019-3>conda.bat activate r-azureml 
Collecting azureml-sdk==1.0.69
  Using cached https://files.pythonhosted.org/packages/04/fa/665cbc4484af86a82663598d21292cfd1e85358dcec6a9369ba82e6c0fdf/azureml_sdk-1.0.69-py3-none-any.whl
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/b0/ee/5ff445dd43b9820e5494d21240e689d3b7cb52bc93f4f164eba84206cd0d/numpy-1.17.4-cp36-cp36m-win_amd64.whl
Collecting azureml-core==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/9a/4f/b5c71c45f9aa82aa2636dd5ec7e19c6c11138c8ef127faa5cbbbc1efa0b7/azureml_core-1.0.69-py2.py3-none-any.whl
Collecting azureml-dataprep[fuse]<1.2.0a,>=1.1.20a
  Using cached https://files.pythonhosted.org/packages/d1/43/ea4dcba146a50bf2dfd4234eca38f241811cd7122154379cad6f5b6967dc/azureml_dataprep-1.1.33-py3-none-any.whl
Collecting azureml-pipeline==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/51/9f/9a5c61cc1284661fa3811c37504ec8704dc6ed758e8d1ad2ad301bd5d085/azureml_pipeline-1.0.69-py3-none-any.whl
Collecting azureml-train==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/90/bf/a0d0805225745042c34436cc59bec7b15c2a2a15789e269265009d0e698c/azureml_train-1.0.69-py3-none-any.whl
Collecting urllib3>=1.23
  Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting ndg-httpsclient
  Using cached https://files.pythonhosted.org/packages/fb/67/c2f508c00ed2a6911541494504b7cac16fe0b0473912568df65fd1801132/ndg_httpsclient-0.5.1-py3-none-any.whl
Collecting azure-mgmt-keyvault>=0.40.0
  Using cached https://files.pythonhosted.org/packages/b3/d1/9fed0a3a3b43d0b1ad59599b5c836ccc4cf117e26458075385bafe79575b/azure_mgmt_keyvault-2.0.0-py2.py3-none-any.whl
Collecting msrestazure>=0.4.33
  Using cached https://files.pythonhosted.org/packages/68/75/5cb56ca8cbc6c5fe476e4878c73f57a331edcf55e5d3fcb4a7377d7d659d/msrestazure-0.6.2-py2.py3-none-any.whl
Collecting contextlib2
  Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Collecting ruamel.yaml<=0.15.89,>=0.15.35
  Using cached https://files.pythonhosted.org/packages/af/b2/367a4a8767ee423690381f15c6c44a76ad3deb5aed042803e7c91c374bc8/ruamel.yaml-0.15.89-cp36-cp36m-win_amd64.whl
Collecting azure-common>=1.1.12
  Using cached https://files.pythonhosted.org/packages/00/55/a703923c12cd3172d5c007beda0c1a34342a17a6a72779f8a7c269af0cd6/azure_common-1.1.23-py2.py3-none-any.whl
Collecting jmespath
  Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting jsonpickle
  Using cached https://files.pythonhosted.org/packages/07/07/c157520a3ebd166c8c24c6ae0ecae7c3968eb4653ff0e5af369bb82f004d/jsonpickle-1.2-py2.py3-none-any.whl
Collecting requests>=2.19.1
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting azure-graphrbac>=0.40.0
  Using cached https://files.pythonhosted.org/packages/3e/93/02056aca45162f9fc275d1eaad12a2a07ef92375afb48eabddc4134b8315/azure_graphrbac-0.61.1-py2.py3-none-any.whl
Collecting azure-mgmt-containerregistry>=2.0.0
  Using cached https://files.pythonhosted.org/packages/97/70/8c2d0509db466678eba16fa2b0a539499f3b351b1f2993126ad843d5be13/azure_mgmt_containerregistry-2.8.0-py2.py3-none-any.whl
Collecting docker
  Using cached https://files.pythonhosted.org/packages/cc/ca/699d4754a932787ef353a157ada74efd1ceb6d1fc0bfb7989ae1e7b33111/docker-4.1.0-py2.py3-none-any.whl
Collecting SecretStorage
  Using cached https://files.pythonhosted.org/packages/82/59/cb226752e20d83598d7fdcabd7819570b0329a61db07cfbdd21b2ef546e3/SecretStorage-3.1.1-py3-none-any.whl
Collecting azure-mgmt-resource>=1.2.1
  Using cached https://files.pythonhosted.org/packages/10/2a/26db9b79f00699e6033f0667dd950085daeb6abf84ae765adfa092f59e37/azure_mgmt_resource-7.0.0-py2.py3-none-any.whl
Collecting msrest>=0.5.1
  Using cached https://files.pythonhosted.org/packages/27/b0/c34b3ea9b2ed74b800520fbefb312cdb7f05c20b8bd42e5e7662a5614f98/msrest-0.6.10-py2.py3-none-any.whl
Collecting six>=1.11.0
  Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting pytz
  Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting pyopenssl
  Using cached https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl
Collecting cryptography!=1.9,!=2.0.*,!=2.1.*,!=2.2.*
  Using cached https://files.pythonhosted.org/packages/c9/ff/43897d282db0fa5c4a34f5413af39a2d1f22c7d47b52d6b3c2fab5b81352/cryptography-2.8-cp36-cp36m-win_amd64.whl
Processing c:\users\kemakino\appdata\local\pip\cache\wheels\62\b8\e1\e2719465b5947c40cd85d613d6cb33449b86a1ca5a6c574269\pathspec-0.6.0-cp36-none-any.whl
Collecting PyJWT
  Using cached https://files.pythonhosted.org/packages/87/8b/6a9f14b5f781697e51259d81657e6048fd31a113229cf346880bb7545565/PyJWT-1.7.1-py2.py3-none-any.whl
Collecting azure-mgmt-storage>=1.5.0
  Using cached https://files.pythonhosted.org/packages/5b/ee/1415439cbbbffce8c9cc9a55b4c0313a2f2a995f9655acfa5aa7344e6d8f/azure_mgmt_storage-7.0.0-py2.py3-none-any.whl
Collecting python-dateutil>=2.7.3
  Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Collecting adal>=1.2.0
  Using cached https://files.pythonhosted.org/packages/4f/b5/3ea9ae3d1096b9ff31e8f1846c47d49f3129a12464ac0a73b602de458298/adal-1.2.2-py2.py3-none-any.whl
Collecting backports.tempfile
  Using cached https://files.pythonhosted.org/packages/b4/5c/077f910632476281428fe254807952eb47ca78e720d059a46178c541e669/backports.tempfile-1.0-py2.py3-none-any.whl
Collecting azure-mgmt-authorization>=0.40.0
  Using cached https://files.pythonhosted.org/packages/5e/17/4724694ddb3311955ddc367eddcd0928f8ee2c7b12d5a6f0b12bca0b03db/azure_mgmt_authorization-0.60.0-py2.py3-none-any.whl
Collecting azureml-dataprep-native<14.0.0,>=13.1.0
  Using cached https://files.pythonhosted.org/packages/d4/b1/6a83ca07608c21ab209961e49bd5f2729107d38f2d59a4af7cc34f23ae15/azureml_dataprep_native-13.1.0-cp36-cp36m-win_amd64.whl
Collecting dotnetcore2>=2.1.9
  Using cached https://files.pythonhosted.org/packages/7f/45/ef5f5609711f6e976ab1e39d03d0c246b6e03cc56942b5e324f58a34bf9b/dotnetcore2-2.1.11-py3-none-win_amd64.whl
Collecting cloudpickle>=1.1.0
  Using cached https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl
Processing c:\users\kemakino\appdata\local\pip\cache\wheels\4c\a5\91\7772af9e21c461f07bb40f26d928d7d231d224977dd8353bab\fusepy-3.0.1-cp36-none-any.whl
Collecting azureml-pipeline-core==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/07/2e/16bdca2fe5319647964d66d7df11ff568e9a184f65531e1d519eacd7992c/azureml_pipeline_core-1.0.69-py2.py3-none-any.whl
Collecting azureml-pipeline-steps==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/1e/61/1903a860b786fd7e7fbae01a814bed5708d6124f3a9e8b88262f79cf355e/azureml_pipeline_steps-1.0.69-py3-none-any.whl
Collecting azureml-train-core==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/99/7d/fa2dcd9dad9c12ee75d02d5937b7300ae861a7ef0b55fc01219de53e08fb/azureml_train_core-1.0.69-py3-none-any.whl
Collecting pyasn1>=0.1.1
  Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting pypiwin32==223; sys_platform == "win32" and python_version >= "3.6"
  Using cached https://files.pythonhosted.org/packages/d0/1b/2f292bbd742e369a100c91faa0483172cd91a1a422a6692055ac920946c5/pypiwin32-223-py3-none-any.whl
Collecting websocket-client>=0.32.0
  Using cached https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl
Collecting jeepney
  Using cached https://files.pythonhosted.org/packages/0a/4c/ef880713a6c6d628869596703167eab2edf8e0ec2d870d1089dcb0901b81/jeepney-0.4.1-py3-none-any.whl
Collecting isodate>=0.6.0
  Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.5.0
  Using cached https://files.pythonhosted.org/packages/a3/12/b92740d845ab62ea4edf04d2f4164d82532b5a0b03836d4d4e71c6f3d379/requests_oauthlib-1.3.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
  Using cached https://files.pythonhosted.org/packages/9e/99/1adcccd6f7761186eb70e4591f45e941529fbf8e5df8e2d79e914887b118/cffi-1.13.2-cp36-cp36m-win_amd64.whl
Collecting backports.weakref
  Using cached https://files.pythonhosted.org/packages/88/ec/f598b633c3d5ffe267aaada57d961c94fdfa183c5c3ebda2b6d151943db6/backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting distro>=1.2.0
  Using cached https://files.pythonhosted.org/packages/ea/35/82f79b92fa4d937146c660a6482cee4f3dfa1f97ff3d2a6f3ecba33e712e/distro-1.4.0-py2.py3-none-any.whl
Collecting azureml-train-restclients-hyperdrive==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/be/be/77659dc5049a57d5114af18f006c7cf9ab883c63f62c51b583bc72134068/azureml_train_restclients_hyperdrive-1.0.69-py3-none-any.whl
Collecting azureml-telemetry==1.0.69.*
  Using cached https://files.pythonhosted.org/packages/e2/2e/bc955dce515bafc0e6edb5af20c83f06edf2551e0592bfdfe9080da96506/azureml_telemetry-1.0.69-py3-none-any.whl
Collecting pywin32>=223
  Using cached https://files.pythonhosted.org/packages/f7/76/29e8e24a4967eb40fdf2f525aae53a1fd6e606382d957aa403f3f12ea235/pywin32-227-cp36-cp36m-win_amd64.whl
Collecting oauthlib>=3.0.0
  Using cached https://files.pythonhosted.org/packages/05/57/ce2e7a8fa7c0afb54a0581b14a65b56e62b5759dbc98e80627142b8a3704/oauthlib-3.1.0-py2.py3-none-any.whl
Processing c:\users\kemakino\appdata\local\pip\cache\wheels\f2\9a\90\de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511\pycparser-2.19-py2.py3-none-any.whl
Collecting applicationinsights
  Using cached https://files.pythonhosted.org/packages/a1/53/234c53004f71f0717d8acd37876e0b65c121181167057b9ce1b1795f96a0/applicationinsights-0.11.9-py2.py3-none-any.whl
Installing collected packages: urllib3, pyasn1, pycparser, cffi, six, cryptography, pyopenssl, ndg-httpsclient, isodate, certifi, chardet, idna, requests, oauthlib, requests-oauthlib, msrest, PyJWT, python-dateutil, adal, msrestazure, azure-common, azure-mgmt-keyvault, contextlib2, ruamel.yaml, jmespath, jsonpickle, azure-graphrbac, azure-mgmt-containerregistry, pywin32, pypiwin32, websocket-client, docker, jeepney, SecretStorage, azure-mgmt-resource, pytz, pathspec, azure-mgmt-storage, backports.weakref, backports.tempfile, azure-mgmt-authorization, azureml-core, azureml-dataprep-native, distro, dotnetcore2, cloudpickle, fusepy, azureml-dataprep, azureml-pipeline-core, azureml-train-restclients-hyperdrive, applicationinsights, azureml-telemetry, azureml-train-core, azureml-pipeline-steps, azureml-pipeline, azureml-train, azureml-sdk, numpy
Successfully installed PyJWT-1.7.1 SecretStorage-3.1.1 adal-1.2.2 applicationinsights-0.11.9 azure-common-1.1.23 
azure-graphrbac-0.61.1 azure-mgmt-authorization-0.60.0 azure-mgmt-containerregistry-2.8.0 azure-mgmt-keyvault-2.0.0 azure-mgmt-resource-7.0.0 azure-mgmt-storage-7.0.0 azureml-core-1.0.69 azureml-dataprep-1.1.33 azureml-dataprep-native-13.1.0 azureml-pipeline-1.0.69 azureml-pipeline-core-1.0.69 azureml-pipeline-steps-1.0.69 azureml-sdk-1.0.69 azureml-telemetry-1.0.69 azureml-train-1.0.69 azureml-train-core-1.0.69 azureml-train-restclients-hyperdrive-1.0.69 backports.tempfile-1.0 backports.weakref-1.0.post1 certifi-2019.11.28 cffi-1.13.2 chardet-3.0.4 cloudpickle-1.2.2 contextlib2-0.6.0.post1 cryptography-2.8 distro-1.4.0 docker-4.1.0 dotnetcore2-2.1.11 fusepy-3.0.1 idna-2.8 isodate-0.6.0 jeepney-0.4.1 jmespath-0.9.4 jsonpickle-1.2 msrest-0.6.10 msrestazure-0.6.2 ndg-httpsclient-0.5.1 numpy-1.17.4 oauthlib-3.1.0 pathspec-0.6.0 pyasn1-0.4.8 pycparser-2.19 pyopenssl-19.1.0 pypiwin32-223 python-dateutil-2.8.1 pytz-2019.3 pywin32-227 requests-2.22.0 requests-oauthlib-1.3.0 ruamel.yaml-0.15.89 six-1.13.0 urllib3-1.25.7 websocket-client-0.56.0

Installation complete.

Also, I can get the current offline run with the following code.

> get_current_run()
<azureml.core.run._OfflineRun>

I'm happy to provide more information if needed.

Error submitting experiment: AttributeError: 'NoneType' object has no attribute 'lower'

With current CRAN release (0.5.6) on Windows, attempting to submit an experiment gets this error when running through the vignette train-and-deploy-to-aci.Rmd.

> run <- submit_experiment(exp, est)
WARNING - 'gpu_support' is no longer necessary; AzureML now automatically detects and uses nvidia docker extension when it is available. It will be removed in a future release.
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  AttributeError: 'NoneType' object has no attribute 'lower'

To Reproduce
Install azuremlsdk from CRAN (v 0.5.6) and attempt to run the vignette train-and-deploy-to-aci.Rmd.

Additional context
This error goes away with version 0.5.7.9000, but providing this issue for the benefit of CRAN users who may experience the error.

Warning: 'gpu_support' is no longer necessary

Describe the bug

Frequently seeing warnings like this:

WARNING - 'gpu_support' is no longer necessary; AzureML now automatically detects and uses nvidia docker extension when it is available. It will be removed in a future release.

Sometimes multiple instances of the warning appear for a single command.

To Reproduce
For example, running samples/training/train-with-tensorflow/train-with-tensorflow.R:

> est <- create_estimator(source_directory = ".", entry_script = "tf_mnist.R",
+                         compute_target = compute_target, cran_packages = c("tensorflow"))
WARNING - 'gpu_support' is no longer necessary; AzureML now automatically detects and uses nvidia docker extension when it is available. It will be removed in a future release.
WARNING - 'gpu_support' is no longer necessary; AzureML now automatically detects and uses nvidia docker extension when it is available. It will be removed in a future release.

Expected behavior
Since the R user isn't passing in gpu_support directly, this warning should be suppressed.

Additional context
I assume the warning is being generated by the underlying Python SDK, since it's not an R-style warning.

Error while trying to use get_workspace() from sdk

Hi,
I have been trying to use the sdk for get the workspace as i want to be able to upload the dataset after R script execution. I have added. It keeps on giving me this error. I really want to use azureml sdk hence have been trying various things for sometime but if it goes like this i may give up on it.
I have checked the logs and couldnt see anything useful. Let me know if any specific log i should look at for the issue.

ModuleExceptionMessage:FailedToEvaluateScript: The following error occurred during script evaluation, please view the output log for more information:
---------- Start of error message from R interpreter ----------
Got exception when invoking script: 'Failed to find execution status file /tmp/tmp74tu03kr/d06ac86b68cb49548f5dd7106936db9f.status'.
---------- End of error message from R interpreter ----------

My designer looks like:

image

and the script inside it is

azureml_main <- function(dataframe1, dataframe2){
if(!require(corrplot)) install.packages("corrplot", repos = "http://cran.us.r-project.org")
if(!require(azuremlsdk)) install.packages("azuremlsdk", repos = "http://cran.us.r-project.org")
library(corrplot)
library(data.table)
azuremlsdk::install_azureml()
library(azuremlsdk)
workspac <- get_workspace("workspacename", "subsid", "resourcename")
return(list(dataset1=dataframe1, dataset2=dataframe2, dataset3=dataframe1))
}

Missing documentation: How to create a dataframe from an Azure SQL Database already registered into the Datastore

I'm trying to create a dataframe from an Azure SQL DB registered into the Datastore.
I'm trying to do that using TabularDatasets with this command:

azureml$core$dataset$Dataset$Tabular$from_sql_query(sqlQuery)

but I'm getting a mysterious issue:

" Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: Invalid argument value or type. Please refer to the documentation for accepted types."

It'd be great to have a companion documentation helping us to do basic operations with the R SDK. For example, just adding the "R" tab to the code chunks in pages like the following one could be the way to go:

Create Azure Machine Learning datasets

[PROPOSAL] Versioning of SDK

I'm concerned it can be very difficult for DRI's to debug/support this SDK. We've had multiple issues in the python SDK where the particular version is important to supporting the issues. For instance, sdk versions 1.0.57-1.0.62 were broken for uploading large models.

I'd like to propose:

  1. Plan to 'release' the R sdk on the same schedule as the python SDK going forward.
  2. Version the R SDK like the python SDK
  3. Hard code version in https://github.com/Azure/azureml-sdk-for-r/blob/master/R/install.R to default to installing the python SDK version which matches the R SDK version.

At the very least, I think 3 would be super helpful. If we move away from the Reticulate-style SDK, we can bump to 2.0 and switch to a sane versioning scheme.

Proxy - self signed certificate - unable to use SDK methods

Describe the bug
When running behind a corporate proxy that is terminating the ssl session and presenting a self signed certificate we are unable to use the sdk components.

To Reproduce
Steps to reproduce the behavior:

  • Have initiating R session behind proxy that is presenting self signed certificate.
  • Try to invoke a method such as get_default_datastore()

Expected behavior
A clear and concise description of what you expected to happen.

  • The ability to trust the self signed certificate (preferred) or
  • Disable verification of presented certificate (obviously risky)

Screenshots

ds <- get_default_datastore(ws)
 Error in py_call_impl(callable, dots$args, dots$keywords) : 
  SSLError: HTTPSConnectionPool(host='centralus.experiments.azureml.net', port=443): Max retries exceeded with url: /discovery (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)'))) 
3.
stop(structure(list(message = "SSLError: HTTPSConnectionPool(host='centralus.experiments.azureml.net', port=443): Max retries exceeded with url: /discovery (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))", 
    call = py_call_impl(callable, dots$args, dots$keywords), 
    cppstack = structure(list(file = "", line = -1L, stack = "C++ stack not available on this system"), class = "Rcpp_stack_trace")), class = c("Rcpp::exception", 
"C++Error", "error", "condition"))) 
2.
workspace$get_default_datastore() 
1.
get_default_datastore(ws) 

Additional context
The specific platform I'm having an issue with is Windows 10 but I dare say the problem appears on on platforms behind a proxy presenting the self signed certificate.

Error in py_call_impl(callable, dots$args, dots$keywords) : AttributeError: 'NoneType' object has no attribute 'startswith'

While going through the tutorial on "train-and-deploy-on-aci.Rmd" in the Azure ML examples, I came across this issue while running the "create_inference_config" chunk:

> inference_config <- inference_config(entry_script = "accident_predict.R",
                                       source_directory = ".",
                                       environment = r_env)

## Error in py_call_impl(callable, dots$args, dots$keywords) : 
## AttributeError: 'NoneType' object has no attribute 'startswith'

I think that this has to do with some version of component in the python back-end but I'm not sure as I cannot find the source code for this.

Could you please enlighten me on the solution? Below you can find the R session details for the RStudio session inside the Azure ML platform.

## R version 3.6.2 (2019-12-12)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 16.04.6 LTS

## Matrix products: default
## BLAS:   /usr/lib/libblas/libblas.so.3.6.0
## LAPACK: /home/azureuser/.local/share/r-miniconda/envs/r-reticulate/lib/libmkl_rt.so

## locale:
## [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8    
## [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8 LC_NAME=C                 
## [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     

## other attached packages:
## [1] azuremlsdk_0.5.7

## loaded via a namespace (and not attached):
## [1] htmlwidgets_1.5.1 compiler_3.6.2    tools_3.6.2       htmltools_0.4.0   rappdirs_0.3.1 reticulate_1.14  
## [7] Rcpp_1.0.3        knitr_1.26        jsonlite_1.6      xfun_0.11         digest_0.6.23     rlang_0.4.2 

Submitting child runs

The R SDK does not support submitting child runs yet whereas the Python SDK does (see here).

As an ML engineer I need to be able to submit child runs so I can better structure the logs of my model trainings.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.