Deep Learning with CNNs Can be done singly or in a group of up to two.

Submission: Put the data and Jupyter notebook files in a folder. Make sure all links to data are relative to the folder so the notebooks can be run.

Classify images with CNNs

Find an image dataset. It cannot be MNIST or CFFAR but can be TMNIST. For TMNIST-Alphabet (94 characters and over 281,000 images) see https://www.kaggle.com/nikbearbrown/tmnist-alphabet-94-characters (Links to an external site.)

Can be done with PyTorch, TensorFlow, and/or Keras

Any dataset other than TMNIST must be pre-approved.

On your Deep Learning model data Change the activation function. How does it affect accuracy? How does it affect how quickly the network plateaus?

Various activation functions:

• Rectified linear unit (ReLU)

• TanH

• Leaky rectified linear unit (Leaky ReLU)

• Parameteric rectified linear unit (PReLU)

• Randomized leaky rectified linear unit (RReLU)

• Exponential linear unit (ELU)

• Scaled exponential linear unit (SELU)

• S-shaped rectified linear activation unit (SReLU)

• Adaptive piecewise linear (APL)

Change the cost function. How does it affect accuracy? How does it affect how quickly the network plateaus?

Various forms of cost:

• Quadratic cost (mean-square error)

• Cross-Entropy

• Hinge

• Kullback–Leibler divergence

• Cosine Proximity

• User-defined

And many more, see https://keras.io/losses/

On your Deep Learning model data

• Change the number of epochs initialization. How does it affect accuracy?

• How quickly does the network plateau?

On your Deep Learning model

Change the gradient estimation. How does it affect accuracy? How does it affect how quickly the network plateaus?

Various forms of gradient estimation:

• Stochastic Gradient Descent

• Adagrad

• RMSProp

• ADAMN

• AGAdadelta

• Momentum

On your Deep Learning model change the network architecture. How does it affect accuracy? How does it affect how quickly the network plateaus?

Various forms of network architecture:

• Number of layers

• Size of each layer

• Connection type

On your Deep Learning model

Change the network initialization. How does it affect accuracy? How does it affect how quickly the network plateaus?

Various forms of network initialization:

• 0

• UniformGaussian

• Xavier

• Glorot Initialization http://andyljones.tumblr.com/post/110998971763/an-explanation-of-xavier-initializationXavier

• Uniform

• Xavier Gaussian

Neural Network Programming - Deep Learning with PyTorch https://youtube.com/playlist?list=PLZbbT5o_s2xrfNyHZsM6ufI0iZENK9xgG (Links to an external site.)

PyTorch & TensorBoard

https://www.youtube.com/watch?v=pSexXMdruFM

(Links to an external site.)

Hyperparameter Tuning And Experimenting - Training Deep Neural Networks

https://deeplizard.com/learn/video/ycxulUVoNbk (Links to an external site.)

Hyperparameter Tuning and Experimenting PyTorch & TensorBoard

https://www.youtube.com/watch?v=ycxulUVoNbk