Benchmark on GPU about mobilenet HOT 5 CLOSED

Hi, wetliu

Someone did it on the iphone7 platform, showing that mobilenet with depthwise-conv is almost 10 times faster than the convolution counterpart. Click here(in Chinese).
I have no knowledge about pytorch, I think it is not so difficult to add a depthwise-conv operator into pytorch if it haven't been implemented.

Best,
Zehao

from mobilenet.

Oh, sorry for the confusion. I am wondering if you also compare mobilenet and its counter part on GPU? Thank you.

from mobilenet.

No.

from mobilenet.

It may be faster in mobile devices using low-end cpus which are more about sequential execution, but going to be (much) slower in high-end / high-parallel processors, like GPU. High-end/parallel processors prefer a large chunk of computation at one time, instead of a butch of small computations (like depth-wise convolutions). This is can the reason why mobilenet doesn't show any real-time speedup. Theoretical speedup (multi-add reduction) is totally different from real-time speedup. You can check figure 1 in this NIPS paper, showing L1 regularization/connection pruning can remove 95% connections in conv layers but even shows down the real-time speed in GPUs.

from mobilenet.

Just to other people who will view this issue in the future, the real problem is not the parallel or sequential computing, but the old implementation of convolution operator through im2col and especially, col2im. These two functions in depthwise convolution are not necessary, and will lead to block divergence on GPU (col2im in the case of backpropagation).

The real speed up on the mobilenet is its 1x1 convolution kernel which takes most of time if the depthwise convolution is well optimized.

@wenwei202 Thank you for your comment, but your L1 example is not a very good example here because either you need to change your convolution connection through a sparseMM operator, or you will have to use GEMM on GPU even most of the connects are sparse.

from mobilenet.