microsoft / cbm Goto Github PK

Cyclic Boosting Machines - an explainable supervised machine learning algorithm

License: MIT License

Python 3.48% C++ 3.09% Jupyter Notebook 93.43%

cbm's Introduction

Cyclic Boosting Machines

This is an efficient and Scikit-learn compatible implementation of the machine learning algorithm Cyclic Boosting -- an explainable supervised machine learning algorithm, specifically for predicting count-data, such as sales and demand.

Features

Optimized for categorical features
Continuous features are discretized using pandas.qcut.
Date auto-expansion (weekday + month).
Feature importance plots: categorical, continuous and interactions.
Metrics to stop training: RMSE, L1, SMAPE.

Usage

The CBM model predicts by multiplying the global mean with each weight estimate for each bin and feature. Thus the weights can be interpreted as % increase or decrease from the global mean. e.g. a weight of 1.2 for the bin Monday of the feature Day-of-Week can be interpreted as a 20% increase of the target.

$\hat{y}_i = \mu \cdot \product^{p}_{j=1} f^k_j$ with $k = \{x_{j,_i} \in b^k_j \}$

pip install cyclicbm

import cbm
from sklearn.metrics import mean_squared_error

# load data using https://www.kaggle.com/c/demand-forecasting-kernels-only
train = pd.read_csv('data/train.csv', parse_dates=['date'])
test  = pd.read_csv('data/test.csv',  parse_dates=['date']) 

# feature engineering
min_date = train['date'].min()

def featurize(df):
    out = pd.DataFrame({
        # TODO: for prediction such features need separate modelling
        'seasonal' : (df['date'] - min_date).dt.days // 60,
        'store'    : df['store'], 
        'item'     : df['item'], 
        'date'     : df['date'],
        # <name-1> _X_ <name-2> to mark interaction features
        'item_X_month': df['item'].astype(str) + '_' + df['date'].dt.month.astype(str)
    })
    
    return out

x_train_df = featurize(train)
x_test_df  = featurize(test)
y_train = train['sales']

# model training
model = cbm.CBM()
model.fit(x_train_df, y_train)

# test on train error
y_pred_train = model.predict(x_train_df).flatten()
print('RMSE', mean_squared_error(y_pred_train, y_train, squared=False))

# plotting
model.plot_importance(figsize=(20, 20), continuous_features=['seasonal'])

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

cbm's People

Contributors

Stargazers

Watchers

Forkers

mindis stefanfroth pietroppeter elb1780 therhaag swenkel annaeg truonghm asokraju dearborn-open-ai

cbm's Issues

minimal example that shows wrong coefficients (1.0 instead of 0 and Inf instead of 2.0) and in general inconsistent behaviour

import numpy as np
import cbm

X = np.array([0, 1]).reshape(-1, 1)
y = np.array([0.0, 1.0])

model = cbm.CBM()  # same result with max_iterations=1
model.fit(X, y)
model.weights  # [[1.0, inf]]

weights in this simple case should be [[0.0, 2.0]] since y_mean is 0.5.

Instead the algorithm outputs 1.0 instead of 0.0 for first weight. (in general it always outputs 1.0 weight for a bin where the target is always 0.0).

It also outputs a very weird inf value for second weight (one can show that this happens after single iteration).

Note that if we change y we may get a correct second weight:

import numpy as np
import cbm

X = np.array([0, 1]).reshape(-1, 1)
y = np.array([0.0, 2.0])

model = cbm.CBM()
model.fit(X, y)
model.weights  # [[1.0, 2.070529847682755]]

link to a public kaggle notebook for reproducibility: https://www.kaggle.com/pietroppeter/issue-cyclic-boosting-machine

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Pip install cyclicbm==0.0.9 fails with gcc version == 11.4

I have been trying to install cyclicbm on my machine and failing to do so (error below). I have created this PR to fix the issue and it's working for me, however since my C/C++ knowledge is very limited, please advise regarding the root cause of the problem. Thank you!

  × Building editable for cyclicbm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [104 lines of output]
      running editable_wheel
      creating /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info
      writing /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/requires.txt
      writing top-level names to /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/top_level.txt
      writing manifest file '/tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/SOURCES.txt'
      reading manifest file '/tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching '*.c' under directory 'src'
      adding license file 'LICENSE'
      writing manifest file '/tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm.egg-info/SOURCES.txt'
      creating '/tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm-0.0.9.dist-info'
      creating /tmp/pip-wheel-9w1cy4ni/.tmp-e50tis6z/cyclicbm-0.0.9.dist-info/WHEEL
      running build_py
      running build_ext
      building 'cbm_cpp' extension
      creating /tmp/tmpdwzr6_2q.build-temp/src
      gcc -pthread -B /home/truonghm/work/simcel/simcel-core/CELPY/Library/.conda/forecaster/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/pybind11/include -I/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/pybind11/include -Isrc -I/home/truonghm/work/simcel/simcel-core/CELPY/Library/.conda/forecaster/include/python3.8 -c src/cbm.cpp -o /tmp/tmpdwzr6_2q.build-temp/src/cbm.o -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++11 -Wall -Wextra -march=native -msse2 -ffast-math -mfpmath=sse -fopenmp -lgomp
      cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
      gcc -pthread -B /home/truonghm/work/simcel/simcel-core/CELPY/Library/.conda/forecaster/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/pybind11/include -I/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/pybind11/include -Isrc -I/home/truonghm/work/simcel/simcel-core/CELPY/Library/.conda/forecaster/include/python3.8 -c src/pycbm.cpp -o /tmp/tmpdwzr6_2q.build-temp/src/pycbm.o -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++11 -Wall -Wextra -march=native -msse2 -ffast-math -mfpmath=sse -fopenmp -lgomp
      cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
      In file included from src/pycbm.h:10,
                       from src/pycbm.cpp:4:
      src/cbm.h: In member function ‘void cbm::CBM::fit_internal(const uint32_t*, const char*, size_t, size_t, size_t, size_t, double, const uint32_t*, double, size_t, size_t, double, bool, float (*)(const uint32_t*, const double*, size_t))’:
      src/cbm.h:134:33: error: ‘numeric_limits’ is not a member of ‘std’
        134 |             double rmse0 = std::numeric_limits<double>::infinity();
            |                                 ^~~~~~~~~~~~~~
      src/cbm.h:134:48: error: expected primary-expression before ‘double’
        134 |             double rmse0 = std::numeric_limits<double>::infinity();
            |                                                ^~~~~~
      Traceback (most recent call last):
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 156, in run
          self._create_wheel_file(bdist_wheel)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
          self._run_build_subcommands()
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
          self.run_command(name)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1233, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
          _build_ext.build_extension(self, ext)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
      /tmp/pip-build-env-x9cbfyq7/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
      !!

Other information

OS: Ubuntu 22.4
gcc version: 11.4
python version: 3.8.16

Prediction give infinite values

Hello,
I am using your implementation to try different features and target values. When I use the predict function to test the performance of the model I receive many Infinite predictions.
There is a way to avoid infinite values as predictions?

Thank you very much for your time and your work.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.