monkeyark / mnist Goto Github PK

Train MNIST dataset with different CNN structures, then export the trained parameters.

W: input width / input size
F: filter width / filter size / kernel size
S: stride
P: padding size on the border
O: output width / output size
O = (W + 2P − F) / S + 1
output size = (input size + 2 * padding - kernel size) / stride + 1

CONV: convolutional layer
FC: fully connected layer / linear layer
input depth = input layer / dimension, MNIST default 1
in_channle = input size / input width, MNIST default 28 (28x28 image size)
out_channel = filter num / output depth / layer / dimension
kernel_size = filter size / filter width
stride = stride
padding = padding

Output width will be floor down to integer if the calculate value is not integer.
As there are only 10 categories for the MNIST dataset, the last FC output neuron is 10.

1 CONV, 2 FC
CONV1, there are 4 filters, with filter size 7, stride 3
Activation function
FC1, output neuron 64
Activation function
FC2, output neuron 10

2 CONV, 1 FC
CONV1, 16 filters, each with filter size 7 and stride 2
Activation function
CONV2, 4 filters, each with filter size 5 and stride 2
Activation function
FC, output neuron is 10

3 CONV, 2 FC
CONV1, 16 filters, filter size 3 and stride 2
Activation function
CONV2, 4 filters, filter size 3 and stride 2
Activation function
CONV3, 16 filters, filter size 3 and stride 1
Activation function
FC1, output neurons 64
FC2, output neuron 10

4 CONV, 2 FC
CONV1, 16 filters, filter size 5, stride 2
Activation function
CONV2, 4 filters, filter size 3, stride 1
Activation function
CONV3, 16 filters, filter size 3, stride 2
Activation function
CONV4, 4 filters, filter size 3, stride 1
FC1, output neuron 64
FC2, output neuron 10