SAGE (Shapley Additive Global importancE) is a game theoretic approach for understanding black-box machine learning models. It summarizes the importance of each feature based on the predictive power it contributes, and accounts for complex feature interactions by using the Shapley value from cooperative game theory.

SAGE is described in detail in this paper, but if you're new to using Shapley values, you might prefer starting with this blog post.

SAGE is model-agnostic, so you can use it with any kind of machine learning model. All you need to do is wrap the model in a function to make it callable, set up a data imputer, and run the sampling algorithm:

import sage

# Get data
x, y = ...

# Get model
model = ...

# Set up imputer for representing data distribution
imputer = sage.utils.MarginalImputer(x, samples=512)

# Set up sampling object
sampler = sage.PermutationSampler(
    model,
    imputer,
    'cross entropy')

# Calculate SAGE values
sage_values = sampler(
    (x, y),
    batch_size=256,
    n_permutations=8096,
    bar=True)

See credit.ipynb for an example using gradient boosting machines (GBM), and bike.ipynb for an example using a PyTorch multi-layer perceptron (MLP).

There are a couple ways to use the code. The easiest way is installing sage-importance with pip:

pip install sage-importance

Alternatively, you can clone the repository and install the package using the local setup.py file:

pip install .

• Ian Covert ([email protected])
• Scott Lundberg
• Su-In Lee

Ian Covert, Scott Lundberg, Su-In Lee. "Understanding Global Feature Contributions Through Additive Importance Measures." arXiv preprint arXiv:2004.00668, 2020.