Training with Sparsity about network-slimming HOT 5 CLOSED

It is better to train from start with sparsity. We haven't tried fine-tuning with sparsity on a pretrained model before.

To implement pruning, you just need to collect all the scaling factors in bn layers. Use model.modules() to go through all the modules.

from network-slimming.

@Eric-mingjie , always thanks for quick answer.
I have implemented InceptionV3prune.py, but found accuracy drops a lot after prune.
The trouble of InceptionV3prune.py is that it has many branchs. So it means the cfg_mask can't be read one by one in turn. My main modification is to solve this problem.

from network-slimming.

The significant drop in accuracy could be due to that the model is not sparse. You can analyze the magnitude of scaling factors in the models.

from network-slimming.

@Eric-mingjie ,yes. The model is not sparse enough.
One more question, druning the sparsity training, the channel_selection is necessary?

from network-slimming.

The use of channel_selection depends on the architecture you use and the way you want to do pruning. For VGG, no channel selection is needed. For ResNet and DenseNet, channel selection is normally needed.

However, if you use mask implementation, channel selection is not needed for any model.

from network-slimming.