This repository contains the code for "CondenseNet: An Efficient DenseNet using Learned Group Convolutions" paper by Gao Huang*, Shichen Liu*, Laurens van der Maaten and Kilian Weinberger (* Authors contributed equally).
CondenseNet is a novel network architecture with unprecedented efficiency. It combines dense connectivity between layers with a mechanism to remove unused connections. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard grouped convolutions โ- allowing for efficient computation in practice. Our experiments demonstrate that CondenseNets are much more efficient than other compact convolutional networks such as MobileNets and ShuffleNets.
Figure 1: Learned Group Convolution with G=C=3.
Figure 2: CondenseNets with Fully Dense Connectivity and Increasing Growth Rate.
- python3
- torch
- ImageNet
As an example, use the following command to train a CondenseNet on ImageNet
python main.py --model condensenet -b 256 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0,1,2,3,4,5,6,7 --resume
As another example, use the following command to train a CondenseNet on CIFAR-10
python main.py --model condensenet -b 256 -j 12 cifar10 \
--stages 14-14-14 --growth 8-16-32 --gpu 0 --resume
We take above trained ImageNet model as an example.
To evaluate the model, use evaluate
to evaluate from given save directory
python main.py --model condensenet -b 16 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate
or use evaluate-from
to evaluate from an arbitrary path
python main.py --model condensenet -b 20 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/BEST/MODEL
However, this model is still large, to convert the model to grouped-convolution version as described in the paper, use convert-from
python main.py --model condensenet -b 20 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--convert-from /PATH/TO/BEST/MODEL
Finally, to directly load from a converted model (CondensedNets), use converted model file and evaluate-from
python main.py --model condensenet_converted -b 16 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/CONVERTED/MODEL
For more examples of usage, refer to script.sh
For detailed options, python main.py --help
Model | FLOPs | Params | CIFAR-10 | CIFAR-100 |
---|---|---|---|---|
CondenseNet-50 | 28.6M | 0.22M | 6.22 | - |
CondenseNet-74 | 51.9M | 0.41M | 5.28 | - |
CondenseNet-86 | 65.8M | 0.52M | 5.06 | 23.64 |
CondenseNet-98 | 81.3M | 0.65M | 4.83 | - |
CondenseNet-110 | 98.2M | 0.79M | 4.63 | - |
CondenseNet-122 | 116.7M | 0.95M | 4.48 | - |
CondenseNet-182* | 513M | 4.2M | 3.76 | 18.47 |
(* trained 600 epochs)
Model | FLOPs | Params | Top-1 Err. | Top-5 Err. | Pytorch Model |
---|---|---|---|---|---|
CondenseNet-74 (C=G=4) | 529M | 4.8M | 26.2 | 8.3 | Download (19M) |
CondenseNet-74 (C=G=8) | 274M | 2.9M | 29.0 | 10.0 | Download (12M) |
[email protected]
[email protected]
We are working on the implementation on other frameworks.
Any discussions or concerns are welcomed!