In this repository, you can find the implementation of different Machine Learning algorithms to do data analysis on a real world dataset.
This work takes place in the context of a project in the Machine Learning course of fall semester 2017 at EPFL. It consists in a Kaggle competition based on the Higgs Boson Machine Learning Challenge (2014).
To complete our analysis, we have been working on a dataset made of two csv files : train.csv and test.csv which both gather information on quantities both raw or computed from the raw quantities.
In the scripts folder you will find:
costs.py: 4 cost functions
- compute_mse: compute mean squared error from error (e)
- compute_mae: compute mean absolute error from error (e)
- compute_loss: compute mse or mae from raw data (y,x,w)
- calculate_loss: compute negative log likelihood
helpers.py: Pre-processing functions and helper functions for each regression method to be implemented
- Pre-processing: standardize, de_standardize, sample_data, build_model_data
- Linear regression using gradient descent: compute_gradient
- Linear regression using stochastic gradient descent: compute_stoch_gradient, batch_iter
- Logistic regression using gradient descent: sigmoid, calculate_gradient, learning_by_gradient_descent
- Logistic regression using newton's method: calculate_hessian, logistic_regression_calculation, learning_by_newton_method
- Penalized logistic regression: penalized_logistic_regression, learning_by_penalized_gradient
proj1_helpers.py: 3 functions to load and write csv files needed for analysis of the data and scoring submissions
- load_csv_data: load csv files
- predict_labels: from computed weights and data outputs the predicted labels
- create_csv_submission: create a csv file with data predictions
implementations.py: 6 functions describing different regression methods for data analysis
- least_squares_GD: Linear regression using gradient descent
- least_squares_SGD: Linear regression using stochastic gradient descent
- least_squares: Least squares regression using normal equations
- ridge_regression: Ridge regression using normal equations
- logistic_regression: Logistic regression using stochastic gradient descent
- reg_logistic_regression: Regularized logistic regression using stochastic gradient descent
project1.ipynb: python notebook used to test our implementations for this project
In the report folder, you will find our report written edited with LaTeX.