Comparison between Caffe and ArmnnCaffeParser processing time about armnn HOT 9 CLOSED

Hi @liviolima80 we're seeing about 5 times faster than upstream Caffe across various tests on Cortex-A72 and Cortex-A54 using the NEON backend. I'm afraid we don't have benchmarking running on Cortex-A9, but realistically a smaller speedup would be expected.

from armnn.

Do you have public benchmark with the comparison to official Caffe library?

from armnn.

Hi @liviolima80 I have had a look and we have no public benchmark data at this time. We're working hard on the machine learning part of https://developer.arm.com/ and I will pass on your request to the relevant teams.

from armnn.

OK @MatthewARM ,
in the meantime, any option to run armnn in multi-thread mode in order to use multiple processors?

from armnn.

Hi @liviolima80 multi-threading is high up on our roadmap. In the meantime, you can try calling the methods of arm_compute::Scheduler yourself. The documentation for that is here: https://arm-software.github.io/ComputeLibrary/v18.03/classarm__compute_1_1_scheduler.xhtml

from armnn.

Hi @MatthewARM ,
I tried to take a look to the documentation and code but it is quite difficult to understand how to use. is it possible to have a clear and easy to use example on how to use the scheduler?
Regards

from armnn.

Please have a look at this question

from armnn.

Hi @AnthonyARM,
thank you for the link. I tried to change the number of threads in the scheduler setting but it seems that the better performance are obtained with the default config that is equal to 4 threads, that corresponds to the number of cores in the Cortex A9 architecture I'm using.
I'm looking forward to see if next releases of armnn will have some further improvement

from armnn.

I'm trying to compare the processing time running a convolutional network with Caffe C++ library and ArmnnCaffeParser built on ARM ComputeLibrary. These are details:

target architecture: Freescale IMX6Q

input image: 256x256 pixel and 3 channels

architecture:
conv2d 3x3 - 32 filters + RELU
conv2d 3x3 - 32 filters + RELU
max pool 2x2
conv2d 3x3 - 64 filters + RELU
conv2d 3x3 - 64 filters + RELU
max pool 2x2
conv2d 3x3 - 128 filters + RELU
max pool 2x2
conv2d 3x3 - 256 filters + RELU
max pool 2x2
conv2d 3x3 - 256 filters + RELU
max pool 2x2
fully connected layer with 48 output
relu
fully connected layer with 3 output
softmax

processing time:

• Caffe c++ library (without ComputeLibrary optimization) : 4.5 seconds
• ArmnnCaffeParser (with ComputeLibrary optimization): 2.2 seconds

I'm a little bit disappointed since my expectations were to boost the processing time more than a factor 2

Do you think that I'm doing something wrong or is this improvement factor comparable with your benchmarks?

Thanks

How did you calculate processing time ?
We are building a caffe2 parser for armnn we want to compare it with the default caffe2

from armnn.