This is Convolutional Neural Network only in python & numpy. It is simple and slow but will get the job done ๐
Weight Initialization : Xavier
Weight Update Policy : ADAM, NAG, Momentum, Vanila
Active Function : ReLU, Sigmoid
Regulization : Droupout(only on fc), L2
Loss Function : Softmax, Logistic
numpy (+ mkl for intel processors. recommend anaconda)
Used sklearn for LabelEncoder & utils.shuffle on examples.
AND gate and CIFAR-10 examples are included.
from classes.neural_net import NeuralNetwork
lr = 1e-4
l2_reg = 5e-6
# 32 * 32 color image
input_size = (3,32,32)
cnn = NeuralNetwork(input_size,
[
{'type': 'conv', 'k': 16, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool'},
{'type': 'conv', 'k': 20, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool'},
{'type': 'conv', 'k': 20, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool'},
{'type': 'output', 'k': len(le.classes_), 'u_type': 'adam'}
]
, lr, l2_reg=l2_reg)
for i in range(600000):
loss, acc = cnn.epoch(train_images, train_labels)
test_loss, test_acc = cnn.predict(train_images, train_labels)
CIFAR-10 example gets ~70% test accuracy in 20 epoch.
classes.NeuralNetwork(input_shape, layer_list, lr, l2_reg=0, dropout_p=1, loss='softmax'):
Parameter | Description |
---|---|
input_shape | Data's numpy shape. |
layer_list | List of layers you want to be networked. All of properties goes to **kwargs. |
lr | Learning rate. |
l2_reg | L2 regularization lambda. |
drouput_p | Dropout rate. 0 < dropout_p <= 1 |
loss | Loss function. 'softmax', 'logistic' |
# type fc, output
classes.NeuralLayer(input_size, k, u_type='adam', a_type='relu')
# type pool
classes.PoolLayer(input_size, f=2, s=2)
# type conv
classes.ConvLayer(input_size, k, f=3, s=1, p=1, u_type='adam', a_type='relu')
Update Policy | u_type |
---|---|
ADAM | 'adam' |
Momentum | 'm' |
Vanilla | 'v' |
NAG | 'nag' |
Activation Function | a_type |
---|---|
ReLU | 'relu' |
Sigmoid | 'sigmoid' |
- More update policies.
- More activation functions.
- Speed tuning.
MIT