Wrap models not machines? about conformalprediction.jl HOT 7 CLOSED

@ablaom currently MLJ models can give deterministic & probabilistic predictions (GLM predicts an entire distribution).
If methods for conformal predictive inference become more developed, can users automatically combine their favorite (appropriate) deterministic predictor w/ their favorite predictive interval method, to make probabilistic predictions?

For example:
currently model = LGBMRegressor() gives deterministic predictions
would a user be able to create a new model composing a deterministic model LGBMRegressor w/ their favorite (appropriate) predictive interval method e.g. conformal(type=naive) to create a new model w/ probabilistic predictions?

Note: at some point it would be great to compare these w/ predictions/Prediction Intervals from NGBoost.py etc

from conformalprediction.jl.

For example:
currently model = LGBMRegressor() gives deterministic predictions
would a user be able to create a new model composing a deterministic model LGBMRegressor w/ their favorite (appropriate) predictive interval method e.g. conformal(type=naive) to create a new model w/ probabilistic predictions?

Yes, I guess that's the design I am suggesting. So, similar to the way BinaryThresholdPredictor wraps any probabilistic predictor and makes it deterministic.

from conformalprediction.jl.

Hi @ablaom 👋🏽 Thanks very much for this suggestion and sorry for the delayed response (been battling Covid this week while also trying to finish my JuliaCon proceedings submission 😅 ). I will implement this first thing once I turn back to working on this package some time next week 👍🏽

from conformalprediction.jl.

Implemented in #10, but will keep this open for now, because I still want to iron out a few things (some related questions here).

from conformalprediction.jl.

#18 should now be strictly in line with MLJ @ablaom, so I will close this. Will still need to figure out how to handle downstream tasks like evaluate in the future.

from conformalprediction.jl.

Did you not need new abstract model subtype(s) at MLJModelInterface? For set-predictions (we already have Interval).

from conformalprediction.jl.

Yup, you're right. Have done that now in #20

from conformalprediction.jl.