-
Numpy-based Net2Net module
- Net2Wider
- Net2Deeper
-
Net2Net using Tensorflow
- Test in MNIST dataset
-
Net2Net core module
- Numpy
- Scipy
-
Tensorflow examples
- Tensorflow
- Slim
-
Baseline architecture
5x5x32(conv1)-pool1-5x5x64(conv2)-pool2-1024(fc1)-10(fc2) -
[EXP 1] Train a teacher network
[Iter: 100] Validation Accuracy : 0.8732 [Iter: 200] Validation Accuracy : 0.9025 [Iter: 300] Validation Accuracy : 0.9313 [Iter: 400] Validation Accuracy : 0.9408 [Iter: 500] Validation Accuracy : 0.9363 [Iter: 600] Validation Accuracy : 0.9466 [Iter: 700] Validation Accuracy : 0.9379 [Iter: 800] Validation Accuracy : 0.9582 [Iter: 900] Validation Accuracy : 0.9583 -
[EXP 2] Train a student network (Net2Wider)
[Iter: 100] Validation Accuracy : 0.9136 [Iter: 200] Validation Accuracy : 0.9689 [Iter: 300] Validation Accuracy : 0.9645 [Iter: 400] Validation Accuracy : 0.9757 [Iter: 500] Validation Accuracy : 0.9762 [Iter: 600] Validation Accuracy : 0.9757 [Iter: 700] Validation Accuracy : 0.9752 [Iter: 800] Validation Accuracy : 0.9765 [Iter: 900] Validation Accuracy : 0.9777 -
[EXP 3] Net2Wider baseline (Random pad)
[Iter: 100] Validation Accuracy : 0.9255 [Iter: 200] Validation Accuracy : 0.9361 [Iter: 300] Validation Accuracy : 0.9418 [Iter: 400] Validation Accuracy : 0.9551 [Iter: 500] Validation Accuracy : 0.9608 [Iter: 600] Validation Accuracy : 0.9653 [Iter: 700] Validation Accuracy : 0.9677 [Iter: 800] Validation Accuracy : 0.9659 [Iter: 900] Validation Accuracy : 0.9690 -
[EXP 4] Train a student network (Net2Deeper)
- Insert a new layer after 'conv1' layer
[Iter: 100] Validation Accuracy : 0.9673 [Iter: 200] Validation Accuracy : 0.9646 [Iter: 300] Validation Accuracy : 0.9718 [Iter: 400] Validation Accuracy : 0.9731 [Iter: 500] Validation Accuracy : 0.9765 [Iter: 600] Validation Accuracy : 0.9612 [Iter: 700] Validation Accuracy : 0.9783 [Iter: 800] Validation Accuracy : 0.9812 [Iter: 900] Validation Accuracy : 0.9785 -
[EXP 5] Net2Deeper baseline (Random initialization)
[Iter: 100] Validation Accuracy : 0.9057 [Iter: 200] Validation Accuracy : 0.9059 [Iter: 300] Validation Accuracy : 0.9446 [Iter: 400] Validation Accuracy : 0.9489 [Iter: 500] Validation Accuracy : 0.9541 [Iter: 600] Validation Accuracy : 0.9581 [Iter: 700] Validation Accuracy : 0.9607 [Iter: 800] Validation Accuracy : 0.9499 [Iter: 900] Validation Accuracy : 0.9663
- All parameters are fixed except new weights from Net2Net.
- The Net2Net core module (net2net.py) can be used in various deep learning libraries (theano, caffe etc.) because it has only numpy dependency.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent