Credit Risk Analysis

Overview

Purpose

Credit risk is an inherently imbalanced problem, as the number of low-risk loans usually outnumber the high-risk ones. Using the credit card dataset provided by LendingClub, we will be applying six different machine learning methods to compare each model’s accuracy, precision, and recall scores in relation to the data’s credit risk. The following analysis will be completed through these tasks:

1. Oversampling the data using the RandomOverSampler and SMORE algorithms.
2. Undersampling the data using the ClusterCentroids algorithm.
3. Using combination (over and under) sampling through the SMOTEEN algorithm.
4. Comparing two machine learning models that are designed to reduce bias: BalancedRandomForestClassifier and EasyEnsembleClassifier.

Resources

• Data source: LoanStats_2019Q1.csv
• Software used: Python, Jupiter Notebook, imbalanced-learn and scikit-learn libraries.

Results

Naive Random Oversampling

• Balanced Accuracy Score: 65.73%

High Risk

• Precision: 1%
• Recall: 71%

Low Risk

• Precision: 100%
• Recall: 60%

SMOTE Oversampling

• Balanced Accuracy Score: 66.22%

High Risk

• Precision: 1%
• Recall: 63%

Low Risk

• Precision: 100%
• Recall: 69%

Undersampling

• Balanced Accuracy Score: 54.47%

High Risk

• Precision: 1%
• Recall: 69%

Low Risk

• Precision: 100%
• Recall: 40%

Combination (Over and Under)

• Balanced Accuracy Score: 64.47%

High Risk

• Precision: 1%
• Recall: 72%

Low Risk

• Precision: 100%
• Recall: 57%

Balanced Random Forest Classifier

• Balanced Accuracy Score: 78.85%

High Risk

• Precision: 30%
• Recall: 70%

Low Risk

• Precision: 100%
• Recall: 87%

Easy Ensemble AdaBoost Classifier

• Balanced Accuracy Score: 93.16%

High Risk

• Precision: 90%
• Recall: 92%

Low Risk

• Precision: 100%
• Recall: 94%

Summary

Out of all the machine learning models tested in this project, the Easy Ensemble AdaBoost Classifier performed the best in all categories of precision, accuracy, and recall. Most of the other models however (excluding the Balanced Random Forest Classifier), had repetitively low precision scores for all high-risk loan data, when compared to the precision scores of low-risk loan data. This disparity in performance makes the other models unreliable when it comes to both sensitivity and precision, which is essential in understanding credit risk. Due to this outcome, I’d recommended only using the Easy Ensemble AdaBoost Classifier for the best results.