design proposal to make nnom better about nnom HOT 2 CLOSED

Hi parai,

Thanks for the suggestion,
I was trying to make NNoM static similar to your idea, and I did try to write it as a static graph at the very beginning, but I gave up.

• If I make it like the code you proposed, the buffer size must be calculated by the user.
• The layer list is not easy to fit different types of layer (dense, conv, relu they all have different parameters, adding new types of layer will affact the old codes).

Those dirty jobs are the ones I tryed hard to avoid to do by myself.

Other than that, the current NNoM can do all the tedious works for you, such as manage the memory, figure out the output shapes in between layers, find out the running order for layers...

The performance impact by NNoM is quite small, memories are pre-allocated and reused by different layers, it takes no time while you run the model.
The running is already as simple as while(layer.next()); takes less than 1us to switch between layers.
It records the order in a shortcut list, and use the list to run the model.

If you put the NNoM code and Keras code side-by-side, they will looks very similar. If you want to fine tune your Keras model by changing a few parameters, you can simply change the parameters in NNoM side at the same position.

Static graphs are great for reducing memory footprint. But they should come with better conversion tools.
If a script can convert your model into C files using CMSIS-NN like they are doing with CMSIS-NN examples, NNoM makes no sense.
However, the problem is writing and modifying those scripts to fit your own built model is already a tedious job. Especially when your model has complex structures such as DenseNet.

I do appreciate your idea,
In one hand, NNoM was designed for fast test on complex model, but the extra memory that it taken is not necessary for the deployment. After compiling, many memory are never used such as the Hook which store the links in between layers. I am thinking of an API to free these kind of memory.
In other hand, NNoM can be easily change to dynamic graph framework in the future if these links are researve.
Just an idea,

Thanks again,
Jianjia

from nnom.

understood, thanks your reply.

from nnom.