WCE is wrong,
FL needs simplifying

Add quadratic weighted kappa loss function for multi-class classification problems with categorical ordinal output.

Potential code implementation: https://www.kaggle.com/chenglongchen/customized-softkappa-loss-in-xgboost

See https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html

We want to be able to do .fit() instead of .train()

damn pycache

When alpha = 1 for WCE, it is better to use in-built Cross Entropy in LightGBM. Add warning for this

If we use a custom classifier metric, but a regular loss function, then we need to disable the sigmoid as it's automatically applied. Should do something like

    def weighted_cross_entropy_metric(yhat, dtrain, alpha=alpha, XGBoost=XGBoost, eval_only=eval_only):

        y = dtrain.get_label()
        if not eval_only:
             yhat = clip_sigmoid(yhat)

See https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html

set alpha=1 for WCE. If it does not equal binary log loss, figure out why

Create a function that uses this

        yhat = 1. / (1. + np.exp(-yhat))

        yhat[yhat >= 1] = 1 - 1e-6

        yhat[yhat <= 0] = 1e-6

instead of copy pasting a milllion times

Currently bokbokbok implements the loss functions to be used in the python api of the main gradient boosting packages.

However, it would be very convenient to add also a version compatible with the sklearn API those packages.

The convention is as follows:

my_amazing_loss_function(y_true, y_pred):

    ... do my magic

   return grad, hess

Do you agree @orchardbirds ?

Basically: