ENH: Limiting number of CPU cores used by shap about shap HOT 4 CLOSED

I just simply ran export OMP_NUM_THREADS=24 in the shell before executing the python script used to calculate shap scores.

from shap.

Xgboost implemented calculation of shap values for us and we basically just call this functionality. Hence it should be possible to limit the number of threads (not cores though) with the global config of xgboost.

import xgboost as xgb

# Show all messages, including ones pertaining to debugging
xgb.set_config(nthread=10, verbosity=2)

# call shap ...

Would be great if you could report back if this helped. This is a bit of a tricky situation for us since we support a number of backends and not all might conform to the same interface. So adding a parameter that limits this might not be available for other libraries etc. Maybe we need to do some digging for catboost, lightgbm and sklearn (sklearn definitely has that).

from shap.

The problem really lies in xgboost's implementation as you mentioned. Setting the OMP_NUM_THREADS environment variable solved my problem. Thank you very much.

By the way, the nthread xgboost config cannot be set with xgb.set_config.

from shap.

Would you mind sharing the code to set the config? Would be great for future reference

from shap.