Add GradientBoostingRegressor about hummingbird HOT 3 CLOSED

I'd enjoy working on this feature if nobody has started on it yet.

from hummingbird.

Go for it!

from hummingbird.

GradientBoostingRegressor has an init argument https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html that specifies the first estimator that is used to initialize the predictions, before the first GB iteration.

I think treelite raises an error when the init argument is different form zero. I haven't checked with them directly but I assume it's because they can only compile trees, not arbitrary estimators.

Would this be an issue with hummingbird as well?

Regarding #64: the HistGradientBoosting estimators don't accept an init parameter, but they do have an internal _baseline_prediction which usually corresponds to the average of the target and which is automatically added to any prediction. This thing isn't a tree and it isn't part of the _predictors list, it's just a private float attribute (EDIT: in multiclass settings it's an array of size n_classes). So maybe #64 will need to take care of that manually?

Alternatively I think we could consider converting _baseline_prediction into a single-node tree in sklearn (I don't think we can do much internally about init though :/)

from hummingbird.