A simple example of a Python web service for real time machine learning model deployment. It is based on this post

This includes Docker integration and SHAP explanations for the deployed model.

Website | Source | Issues

• docker
• docker-compose (Recommended)

Make sure that you have a model in the main directory. You can launch the example using the following line in order to create a quick classification model.

$ python ./example/build_linear_binary.py

or one of the scripts in the ./example folder

• variables.env: Controls API parameters via environment variables

• requirements.txt: Controls Python packages installed inside the container

• model.joblib: Model saved inside a dictionary with this format

  {
      "model": trained_model,
      "metadata": {"features": [
          {"name": "feature1", "type": "numeric", "accepts_missing": True},
          {"name": "feature2", "type": "numeric", "default": -1, "accepts_missing": False},
          {"name": "feature3", "type": "category", "categories": ["A", "B"], "accepts_missing": True}]}
  }

Build the image (this has to be done every time the code or the model changes)

$ docker-compose build

Create and run the container

$ docker-compose up

Create the environment

$ conda create -n flask_ml python=3
$ conda activate flask_ml

Install requirements

$ pip install -r ./requirements-service.txt  
$ pip install -r ./requirements.txt  

Run the API service

$ python service.py  

This example considers that the API was launched locally without docker and with the default parameters (localhost at port 5000) and its calling the example model.

For /predict endpoint the JSON string in the payload of hte request can take two forms:

1. The first, the payload is a record or a list of records with one value per feature. This will be directly interpreted as the input for the model.

2. The second, the payload is a dictionary with 1 or 2 elements. The key _data is mandatory because this will be the input for the model and its format is expected to be a record or a list of records. On the other hand, the key _samples (optional) will be used to obtain different explanations.

If _samples is not given, then the explanations returned are the raw output of the trees, which varies by model (for binary classification in XGBoost this is the log odds ratio). On the contrary, if _samples is given, then the explanations are the output of the model transformed into probability space (note that this means the SHAP values now sum to the probability output of the model). See the SHAP documentation for details.

Endpoint: /health

$ curl -X GET http://localhost:5000/health
up

Endpoint: /ready

$ curl -X GET http://localhost:5000/ready
ready

Endpoint: /service-info

$ curl -X GET http://localhost:5000/service-info

{
  "debug": true,
  "running-since": 1563355369.6482198,
  "serving-model-name": "model.joblib",
  "serving-model-type": "SKLEARN_MODEL",
  "version-template": "2.2.0"
}

Endpoint: /info

$ curl -X GET http://localhost:5000/info

{
  "metadata": {
    "features": [
      {
        "default": -1,
        "importance": 0.2,
        "name": "feature1",
        "type": "numeric"
      },
      {
        "default": -1,
        "importance": 0.1,
        "name": "feature2",
        "type": "numeric"
      },
      {
        "default": -1,
        "importance": 0.3,
        "name": "feature3",
        "type": "numeric"
      }
    ]
  },
  "model": {
    "type": "<class 'sklearn.ensemble.forest.RandomForestClassifier'>",
    "predictor_type": "<class 'sklearn.ensemble.forest.RandomForestClassifier'>",
    "is_explainable": false,
    "task": "BINARY_CLASSIFICATION",
    "class_names": ["0", "1"]
  }
}

Endpoint: /predict

$ curl -d '[{"feature1": 1, "feature2": 1, "feature3": 2}, {"feature1": 1, "feature2": 1, "feature3": 2}]' -H "Content-Type: application/json" -X POST http://localhost:5000/predict

{
  "prediction": [0, 0]
}

Endpoint: /predict?proba=1

$ curl -d '{"feature1": 1, "feature2": 1, "feature3": 2}' -H "Content-Type: application/json" -X POST "http://localhost:5000/predict?proba=1"

{
  "prediction": [{
    "0": 0.8,
    "1": 0.2
  }]
}

Endpoint: /features

$ curl -X GET "http://localhost:5000/features"

[
  {
    "default": -1,
    "importance": 0.2,
    "name": "feature1",
    "type": "numeric"
  },
  {
    "default": -1,
    "importance": 0.1,
    "name": "feature2",
    "type": "numeric"
  },
  {
    "default": -1,
    "importance": 0.3,
    "name": "feature3",
    "type": "numeric"
  }
]

Endpoint: /predict?proba=1&explain=1

$ curl -d '{"feature1": 1, "feature2": 1, "feature3": 2}' -H "Content-Type: application/json" -X POST "http://localhost:5000/predict?proba=1&explain=1"

{
  "explanation": {
    "feature1": 0.10000000149011613,
    "feature2": 0.03333333383003871,
    "feature3": -0.1666666691501935
  },
  "prediction": [{
    "0": 0.7,
    "1": 0.3
  }]
}