kasaai / explain-ml-pricing Goto Github PK
View Code? Open in Web Editor NEWTowards Explainability of Machine Learning Models in Insurance Pricing
Towards Explainability of Machine Learning Models in Insurance Pricing
While we're targeting Variance, it would be helpful for the broader community to parse the paper if we qualify that exams/ASOPs are for CAS/US. Even better if we can cite some international societies' efforts.
From the ratemaking section
3. Perform analysis on the data, employing desired method or methods to estimate
needed rate relativites
4. Select final rate relativities based on rate indications
5. Present rates to the reglator, including explanation of the steps followed to derive
the rates
6. Answer questions from regulators regarding the method employed
The focus of this paper is on steps 5 and 6
I don't think we can get at relativities with ML models, so maybe we need to revise this bit to say 4-5 will be different in the world with ML models. It may also make sense to point out somewhere that we'll be using ML as a drop-in replacement for GLM rather than for feature engineering only.
This is an open-ended question that I mentioned on our call. Is it realistic to assume that there could be a single standardized set of questions that, properly answered, could reasonably qualify any model?
For example, suppose the question were simply:
"Demonstrate that the rates produced by the model are not inadequate, excessive, or unfairly discriminatory."
That would technically address any concern insofar as a successful answer would mean the model could be approved, but realistically speaking, chances are that it would not produce good regulatory outcomes since modelers wouldn't really know how to answer that.
What I mean to say is that I guess after reading the paper over, I prefer to think of the "question and answer" framework in terms of a set of idealized questions and that a perfect set of questions may not exist that cover every model. My sense has been that no matter how detailed your questions and how varied, you'll probably see some model that leaves you with additional questions from time to time.
That is, I see the question-and-answer thing more as a metaphor for how the actuary should conceptualize the requirement to communicate to intended users of a model than as a suggestion to actually come up with a list of specific questions.
IF you agree - and I suppose that could be a big if - then we may want to consider rewording some of the language around the "standardized set of questions" a bit accordingly.
since rugs are no more
Reminder to point this out when talking about interpretability/transparency... Linear models are "interpretable", but if you have a high dimensional GLM with a bunch of interactions or variable transformations, you can quickly lose that.
Because many machine learning models are deterministic, they may not admit of standard metrics for model comparison (e.g., it’s not straightforward to calculate an AIC over a neural network).
We'll want to reword this e.g. "does not assume an underlying stochastic process" since deterministic has a different meaning in ML
When run the R codes:
fi <- ingredients::feature_importance(
explainer_nn,
loss_function = function(observed, predicted, weights) {
sqrt(
sum(((observed - predicted) ^ 2 * weights) /sum(weights))
)
},
weights = testing_data$exposure,
variables = predictors,
)
the result is “Error in loss_function(observed, predict_function(x, sampled_data)) : argument "weights" is missing, with no default”.
The problem should be “loss_function”,but I don’t know why.
one of the social criteria for rating variables is controllability of premiums by policyholders, we may want to call this out somewhere when discussing questions to ask #8
Some candidates
Brief overview of the ratemaking workflow
Is it accurate to say GLM is state of the art for risk classification today? Are there sources we can/should cite or does it go without saying?
literature review for ML interpretability would be included here, also defining interpretability for the purpose of the paper
———. 2018. “Regulatory Review of Predictive Models 10/25/18 Exposure Draft.” Casualty Actuarial; Statistical (C) Task Force.
https://www.naic.org/documents/cmte_c_catf_
exposure_predictive_model_white_paper.pdf.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.