This repository contains notebooks that illistrate different machine learning algorithms using embedPy.
- Neural networks (NN notebook)
- Dimensionality Reduction
- K-nearest Neighbours
- Feature Engineering (NN notebook)
- Decision Trees
- Random Forests
If you have Docker installed you can create a directory called q
and place your kc.lic
(or k4.lic
) and l64.zip
files into a q
directory and run:
docker run --rm -it -v `pwd`/q:/tmp/q -p 8888:8888 kxsys/ml_notebook_examples
Now point your browser at http://localhost:8888/tree/notebooks
N.B. build instructions for the image are available
These are the steps that are required to run the notebooks:
-
Install Q (version 3.5 or higher/64-bit)
- make sure q is on the path
- set QHOME
- also see https://code.kx.com/q/tutorials/install/
-
Install Anaconda-Python(version 3.5 or higher)
Steps 3 and 4 are only required to run Neural Network Notebooks
-
Install Cuda (version 9.0)
-
Install tensorflow
-
Install embedPy
-
Install jupyterq
- see https://github.com/KxSystems/jupyterq
- Check if the kdb+ notebook works
-
Install required python packages
-
Common modules in Graphics.q
- numpy
- matplotlib
-
Neural Networks:
- Keras
- scikit-learn
-
Dimensionality reduction:
- numpy
- matplotlib
- Keras
- scikit-learn
-
K-Nearest Neighbours:
- scikit-learn
- numpy
- matplotlib
-
Feature Engineering:
- scikit-learn
- Keras
- numpy
- matplotlib
-
Decision trees:
- numpy
- scipy
- graphviz
- matplotlib
- scikit_learn
- xgboost
-
Random Forest:
- numpy
- pandas
- scikit_learn
- xgboost
To install these packages run this line in the terminal:
pip install -r requirements.txt
-