This tutorial covers the basics of Deep Learning with Convolutional Neural Nets. The tutorial is broken into three notebooks. The topics covered in each notebook are:

1. Intro.ipynb:

   • Linear Regression as single layer, single neuron model to motivate the introduction of Neural Networks as Universal Approximators that are modeled as collections of neurons connected in an acyclic graph
   • Convolutions and examples of simple image filters to motivate the construction of Convolutionlal Neural Networks.
   • Loss/Error functions, Gradient Decent, Backpropagation, etc

2. Mnist.ipynb:

   • Visualizing Data
   • Constructing simple Convolutional Neural Networks
   • Training and Inference
   • Visualizing/Interpreting trained Neural Nets

3. CIFAR-10.ipynb:

   • Data Generators
   • Overfitting
   • Data Augmentation

The tutorial is aimed at new hal system users. Below are brief instructions for getting started with HAL to run these notebooks.

Brief description: "My name is HAL. I became operational on March 25 2019 at the Innovative Systems Lab in Urbana, Illinois. My creators are putting me to the fullest possible use, which is all I think that any conscious entity can ever hope to do." (paraphrazed from https://en.wikipedia.org/wiki/HAL_9000)

To request access to the system, please see the official wikipage: https://wiki.ncsa.illinois.edu/display/ISL20/HAL+cluster

Quick Setup for this tutorial:

a) Log in to HAL:

• ssh into hal: ssh <netid>@hal.ncsa.illinois.edu
• Load PowerAI: module load powerai

Now clone the PowerAI conda envirnment (we'd need to install additional packages):

• conda create --name [your_env_name] --clone powerai_env

Next activate your new environment and install keras, opencv, and jupyter.

b) git clone this repo to your home dir

c) Now let's run an interactive session with Jupyter Notebook:

• Start an interaction session:

  swrun -p gpux1

• Start a jupyter session on the compute node:

  unset XDG_RUNTIME_DIR

  jupyter notebook --ip=0.0.0.0 --port=6006

• Tunnel into the compute node from local machine:

  ssh -L 8888:<remote_machine>:6007 <netid>@hal.ncsa.illinois.edu

• Direct your webbrowser to: https://localhost:8888. It will prompt for a pass code, which can be found at $HOME/.jupyter/.jupyter_pass

To learn more about how to use HAL see the slides here and the official wikipage

References:

The code examples presented here are mostly taken (verbatim) or inspired from the following sources. I made this curation to give a quick exposure to very basic but essential ideas/practices in deep learning to get you started fairly quickly, but I recommend going to some or all of the actual sources for an in depth survey:

• CS231n: Convolutional Neural Networks for Visual Recognition
• Deep Learnig Specialization, Andrew Ng
• PyTorch Challenge, Udacity
• Deep Learning with Python
• Keras Blog