This project shows how to deploy a Fashion MNIST MLflow model using an online managed endpoint. Endpoint 1 demonstrates the simplest scenario, endpoint 2 shows how to wrap the deployment with custom code, endpoint 3 exemplifies "aml_token" authentication, and endpoint 4 contains an example of blue-green deployments.
To learn more about the code in this repo, check out the accompanying blog post: https://bea.stollnitz.com/blog/aml-online-endpoint/
- You need to have an Azure subscription. You can get a free subscription to try it out.
- Create a resource group.
- Create a new machine learning workspace by following the "Create the workspace" section of the documentation. Keep in mind that you'll be creating a "machine learning workspace" Azure resource, not a "workspace" Azure resource, which is entirely different!
- If you have access to GitHub Codespaces, click on the "Code" button in this GitHub repo, select the "Codespaces" tab, and then click on "New codespace."
- Alternatively, if you plan to use your local machine:
- Install the Azure CLI by following the instructions in the documentation.
- Install the ML extension to the Azure CLI by following the "Installation" section of the documentation.
- Install and activate the conda environment by executing the following commands:
conda env create -f environment.yml conda activate aml_online_endpoint
- In a terminal window, log in to Azure by executing
az login --use-device-code
. - Set your default subscription by executing
az account set -s "<YOUR_SUBSCRIPTION_NAME_OR_ID>"
. You can verify your default subscription by executingaz account show
, or by looking at~/.azure/azureProfile.json
. - Set your default resource group and workspace by executing
az configure --defaults group="<YOUR_RESOURCE_GROUP>" workspace="<YOUR_WORKSPACE>"
. You can verify your defaults by executingaz configure --list-defaults
or by looking at~/.azure/config
. - You can now open the Azure Machine Learning studio, where you'll be able to see and manage all the machine learning resources we'll be creating.
- Install the Azure Machine Learning extension for VS Code, and log in to it by clicking on "Azure" in the left-hand menu, and then clicking on "Sign in to Azure."
- Under "Run and Debug" on VS Code's left navigation, choose the "Train endpoint 1 locally" run configuration and press F5. An 'endpoint_1/model' folder is created with the trained model.
- Repeat for all other endpoints.
- You can analyze the metrics logged in the "mlruns" directory with the following command:
mlflow ui
- Make a local prediction using the trained mlflow model. You can use either csv or json files:
cd aml_online_endpoint/endpoint_1
mlflow models predict --model-uri "model" --input-path "../test_data/images.csv" --content-type csv --env-manager local
mlflow models predict --model-uri "model" --input-path "../test_data/images.json" --content-type json --env-manager local
This same syntax should work for all endpoint with the exception of endpoint 2. Here are the commands for endpoint 2:
cd aml_online_endpoint/endpoint_2
mlflow models predict --model-uri "pyfunc_model" --input-path "../test_data/images.csv" --content-type csv --env-manager local
mlflow models predict --model-uri "pyfunc_model" --input-path "../test_data/images.json" --content-type json --env-manager local
cd endpoint_1
Create the model resource on Azure ML.
az ml model create --path model/ --name model-online-1 --version 1 --type mlflow_model
Create the endpoint.
az ml online-endpoint create -f cloud/endpoint.yml
az ml online-deployment create -f cloud/deployment.yml --all-traffic
Invoke the endpoint.
az ml online-endpoint invoke --name endpoint-online-1 --request-file ../test_data/images_azureml.json
Clean up the endpoint, to avoid getting charged.
az ml online-endpoint delete --name endpoint-online-1 -y
cd ../endpoint_2
Create the model resource on Azure ML.
az ml model create --path pyfunc_model/ --name model-online-2 --version 1 --type mlflow_model
Create the endpoint.
az ml online-endpoint create -f cloud/endpoint.yml
az ml online-deployment create -f cloud/deployment.yml --all-traffic
Invoke the endpoint.
az ml online-endpoint invoke --name endpoint-online-2 --request-file ../test_data/images_azureml.json
Invoke the endpoint using a curl command.
chmod +x invoke.sh
./invoke.sh
Clean up the endpoint, to avoid getting charged.
az ml online-endpoint delete --name endpoint-online-2 -y
cd endpoint_3
Create the model resource on Azure ML.
az ml model create --path model/ --name model-online-3 --version 1 --type mlflow_model
Create the endpoint.
az ml online-endpoint create -f cloud/endpoint.yml
az ml online-deployment create -f cloud/deployment.yml --all-traffic
Invoke the endpoint.
az ml online-endpoint invoke --name endpoint-online-3 --request-file ../test_data/images_azureml.json
Clean up the endpoint, to avoid getting charged.
az ml online-endpoint delete --name endpoint-online-3 -y
cd endpoint_4
Create the model resource on Azure ML.
az ml model create --path model/ --name model-online-4 --version 1 --type mlflow_model
Create the endpoint.
az ml online-endpoint create -f cloud/endpoint.yml
az ml online-deployment create -f cloud/deployment-blue.yml --all-traffic
az ml online-deployment create -f cloud/deployment-green.yml
az ml online-endpoint update --name endpoint-online-4 --traffic "blue=90 green=10"
Invoke the endpoint.
az ml online-endpoint invoke --name endpoint-online-4 --request-file ../test_data/images_azureml.json
Clean up the endpoint, to avoid getting charged.
az ml online-endpoint delete --name endpoint-online-4 -y