pulp-platform / dory Goto Github PK

A tool to deploy Deep Neural Networks on PULP-based SoC's

License: Apache License 2.0

Python 47.87% C 47.70% Shell 2.33% Dockerfile 0.25% Turing 0.88% Perl 0.96%

dory's Issues

Diana_TVM: element-wise sum output transfers are not tiled?

@ABurrello for Diana_TVM I noticed while going through both the convolution and the addition template that the output/last DMA call in the tiling loop does not seem to be tiled for the addition (it is tiled for the convolution), and as such it seems to me that it will always write to the same memory address in the output.

I.e. for the final DMA call, there's no dory_get_tile_3d() and l2_y is set directly in the DMA call field ( DMA_copy_y.ext = l2_y; instead of DMA_copy_y.ext = l2_y_tile;).
Is it possible that this will result in an error if the tensor addition gets tiled?

Can you double check this please? Thanks!

Repository pulp-nn-1d and pulp-nn-mixed not found!

when trying to clone dory with its submodules I get the following error from git:

Cloning into '/home/quentin/dory/pulp-nn-1d'...
Username for 'https://github.com': ESEQU
Password for 'https://[email protected]':
remote: Repository not found.
fatal: repository 'https://github.com/pulp-platform/pulp-nn-1d/' not found
fatal: clone of 'https://github.com/pulp-platform/pulp-nn-1d' into submodule path '/home/quentin/dory/pulp-nn-1d' failed
Failed to clone 'pulp-nn-1d'. Retry scheduled

That is because the respective repositories do not exist. I also could not find them anywhere else on github.
Can I still use DORY without them or are they crusial for all functionality?
Also how can I get them?
Thanks in advance

Failed to run dronet example

I used the example here: https://github.com/pulp-platform/dory_examples/tree/60dd66bf1d9ce81c0cdd5f70e51040a332dce8e5/examples/Nemo_examples/8-bits-2D/dronet_complete

and I ran
python3 network_generate.py NEMO PULP.PULP_gvsoc./dory/dory_examples/config_files/config_NEMO_dronet.json

it gives this error:

Insert tiling parameters per layer inside graph nodes
RuntimeWarning: Unexpected end-group tag: Not all data was converted
return _pywrapcp.Solver_DefaultSolverParameters()
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0308 13:22:36.521386 1 constraint_solver.cc:1408] Check failed: parameters.array_split_size() > 0 (0 vs. 0) Were parameters built using Solver::DefaultSolverParameters() ?
*** Check failure stack trace: ***
Abort trap: 6

Any idea what might went wrong? Thanks for the help!

Dory error during converting nemo's output onnx file

hi @ABurrello

with the help of nemo I export the onxx file of my model but I got this error during converting onnx file to C code by dory :

here is my onnx model :

Best,
Milad

Question about the width of Bn_Relu_Bits

Hi!
I noticed that the PenguiNet has two versions in different width of Bn_Relu_Bits, one is 32 and the other is 64.

I wonder that what is the difference between them and if there is something to notice when generating an ONNX quantized network with the Nemo tool if I want the Bn_Relu_Bits to be 32 instead of 64？

when I use DORY to deploy the yolo-v3 tiny backbone(Purely feed forward network) , if the Bn_Relu_Bits is 32 the checksum failed and if the Bn_Relu_Bits is 64 the checksum is OK. Is there something wrong with my quantization process？

Best regards,
Zhu Yuqing

json files don't get installed with `pip install .`

  File "/tvm-fork/python/tvm/relay/backend/contrib/soma_dory/codegen.py", line 201, in soma_dory_compiler
    converter = onnx_manager(codegen.dory_graph, config_file, '')
  File "/usr/local/lib/python3.8/dist-packages/dory/Hardware_targets/Diana/Diana_TVM/HW_Parser.py", line 42, in __init__
    with open(os.path.join(file_path, "pattern_rules.json")) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/dory/Hardware_targets/Diana/Diana_TVM/pattern_rules.json'

I installed this with pip install . from the root of the repository
Current workaround is to install with pip install -e

@ABurrello maybe this is due to an issue with setup.py?

CC @maartenvds

mixed precision

Hi,

What happened to pulp-nn-mixed directory? I want to use a mixed-prec. quantized network if possible. Thanks.

Best regards,
Beran

Error when using Dory to deploy modified-DroNet.

Hi,there
I made the following modifications to Dronet: add a fully connected layer self.fc2 = nn.Linear(in_features=fc_size, out_features=3, bias=False) in Dronet, where fc_size = 128*7*7.

x = x.flatten(1)
x_copy = x       
x = self.fc1(x) 
steer = x[:, 0]
coll = self.sig(x[:, 1]) 
x_copy = self.fc2(x_copy)
x_copy = self.relu1(x_copy)
sign = self.softmax(x_copy)    # [a,b,c] #a+b+c = 1

return [steer,coll, sign]

which mad the output of dronet like this:

but I meet following Error:

Traceback (most recent call last):
  File "network_generate.py", line 102, in <module>
    main()
  File "network_generate.py", line 81, in main
    PULP_Nodes_Graph = onnx_m('GAP8', args.chip, args.network_dir + net).parameters_from_onnx(100)
  File "../ONNX_management.py", line 471, in parameters_from_onnx
    new_node = self.create_node(node_element(), first_node, node_iterating, model, PULP_Nodes_Graph)
  File "../ONNX_management.py", line 247, in create_node
    temp = temp.reshape(temp.shape[0], PULP_Nodes_Graph[-1].output_channels, PULP_Nodes_Graph[-1].output_h, PULP_Nodes_Graph[-1].output_w)
ValueError: cannot reshape array of size 18816 into shape (3,2,1,1)

I add some print for Debug,and got:

Flatten_145
MatMul_146
         MatMul_146
(2, 6272)
2 128 7 7
         1 1
Constant_147
Gather_148
Constant_149
Gather_150
Sigmoid_151
MatMul_152
         MatMul_152
(3, 6272)
3 2 1 1

I guess MatMul_152 take the output of MatMul_146(output_w=1,output_h =1) as input which result in this ValueError.
But according to the design of my network, it is planned to use 128*7*7 as the input of MatMul_152.
So, How can I solve this Error?
Hoping to hearing from you
Regards,
Mathilda

Target HW

Is the DORY framework only compatible with commercial MCUs like GAP-8 as the target hardware, or can it also be used with custom architectures based on other PULP platforms?

How to get the .json as well as .txt files needed to deploy the model ?

I have successfully quantized the model with the NEMO tool and generated the .onnx file, but when generating the network with the "network_generate.py" file, I get an error: "out_layer0.txt not found." How do I get these .txt files for deploying the model, and I was wondering if the .json file can be generated automatically? Or do I have to fill them in manually? I would be very grateful for your reply, the job is a great one.

`build_docker.sh` is broken

Not sure whether you're accepting third-party contributions or not, but the case is too simple so hopefully an issue suffices:

In docker_utils/build_docker.sh, docker -t must be switched to docker build -t.

(bug found on Ubuntu 20.04, Docker 24.0.6)

Is Dory supports residual connections networks which has multi-output now?

@ABurrello I asked this question #15 before . And you said that Dory will support the multi-output model based on the residual network in the future. Do you have any progress now?

dory_examples, make all

Hi,

In dory_examples, after I run python3 network_generate.py to generate the files, in application dir., I run make clean all run CORE=8 platform=gvsoc and get the following error:

DORY_network/inc/mchan_test.h: In function 'mchan_transfer':
DORY_network/inc/mchan_test.h:105:54: error: 'MCHAN_CMD_CMD_TYPE_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_LEN_BIT'?
  *(volatile int*) MCHAN_COMMAND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                      ^~~~~~~~~~~~~~~~~~~~~~
                                                      MCHAN_CMD_CMD_LEN_BIT
DORY_network/inc/mchan_test.h:105:54: note: each undeclared identifier is reported only once for each function it appears in
DORY_network/inc/mchan_test.h:105:88: error: 'MCHAN_CMD_CMD_INC_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_LEN_BIT'?
  *(volatile int*) MCHAN_COMMAND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                        ^~~~~~~~~~~~~~~~~~~~~
                                                                                        MCHAN_CMD_CMD_LEN_BIT
DORY_network/inc/mchan_test.h:105:124: error: 'MCHAN_CMD_CMD__2D_EXT_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_TYPE_BIT'?
  *(volatile int*) MCHAN_COMMAND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                                                            ^~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                                                            MCHAN_CMD_CMD_TYPE_BIT
DORY_network/inc/mchan_test.h:105:159: error: 'MCHAN_CMD_CMD_ELE_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_LEN_BIT'?
  *(volatile int*) MCHAN_COMMAND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                                                                                               ^~~~~~~~~~~~~~~~~~~~~
                                                                                                                                                               MCHAN_CMD_CMD_LEN_BIT
DORY_network/inc/mchan_test.h:105:191: error: 'MCHAN_CMD_CMD_ILE_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_ELE_BIT'?
  *(volatile int*) MCHAN_COMMAND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                                                                                                                               ^~~~~~~~~~~~~~~~~~~~~
                                                                                                                                                                                               MCHAN_CMD_CMD_ELE_BIT
DORY_network/inc/mchan_test.h:105:223: error: 'MCHAN_CMD_CMD_BLE_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD_ILE_BIT'?
 ND_QUEUE = len | (type<<MCHAN_CMD_CMD_TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                                                                                                                                  ^~~~~~~~~~~~~~~~~~~~~
                                                                                                                                                                                                                               MCHAN_CMD_CMD_ILE_BIT
In file included from DORY_network/inc/dory.h:20:0,
                 from DORY_network/src/network.c:22:
DORY_network/inc/mchan_test.h:105:261: error: 'MCHAN_CMD_CMD__2D_TCDM_BIT' undeclared (first use in this function); did you mean 'MCHAN_CMD_CMD__2D_EXT_BIT'?
 TYPE_BIT) | ( incr<<MCHAN_CMD_CMD_INC_BIT) | (twd_ext <<MCHAN_CMD_CMD__2D_EXT_BIT) | (ele<<MCHAN_CMD_CMD_ELE_BIT) | (ile <<MCHAN_CMD_CMD_ILE_BIT) | (ble <<MCHAN_CMD_CMD_BLE_BIT) | (twd_tcdm << MCHAN_CMD_CMD__2D_TCDM_BIT);
                                                                                                                                                                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                                                                                                                                                                                                     MCHAN_CMD_CMD__2D_EXT_BIT
/home/kilic/Documents/gap_sdk/tools/rules/pulp_rules.mk:173: recipe for target '/home/kilic/Documents/thesis/gap/dory/dory_examples/application/BUILD/GAP8_V2/GCC_RISCV/DORY_network/src/network.o' failed
make: *** [/home/kilic/Documents/thesis/gap/dory/dory_examples/application/BUILD/GAP8_V2/GCC_RISCV/DORY_network/src/network.o] Error 1

Anybody knows about this issue? Thanks.

Best regards,
Beran

dory$ git submodule update --init --recursive

uie24011@ozd1161u:~/Desktop/SDK/pulp-sdk/dory$ git submodule update --init --recursive
Cloning into '/home/uie24011/Desktop/SDK/pulp-sdk/dory/pulp-nn-1d'...
Username for 'https://github.com': nagatejapallapu
Password for 'https://[email protected]':
remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.
fatal: Authentication failed for 'https://github.com/pulp-platform/pulp-nn-1d/'
fatal: clone of 'https://github.com/pulp-platform/pulp-nn-1d' into submodule path '/home/uie24011/Desktop/SDK/pulp-sdk/dory/pulp-nn-1d' failed
Failed to clone 'pulp-nn-1d'. Retry scheduled

Did DORY can be used in any Net?

Hi,
I noticed that in the readme file, the Limitations Part mentioned:The DORY framework is currently tested on feed-forward networks with single-wire residual connections.

I want to know that can Dory be used in any Neural Networks or other Deep Learning Networks ？

Modifications about Dory

Hi，
I noticed that you made some changes to the Dory repository and rebuilt a new repository for dory_example.
Will this improve the efficiency of the chip? By the way, I tried to deploy pulp-dronet-V2 to GAP8 with Dory. Will the modification of Dory has any affection on the deployment of pulp-dronet-V2?
Regards,
Mathilda.

building docker scripts is not working

this script has a problem
./docker_utils/build_docker.sh $IMAGE_NAME

`05_DORY_Frontend_final_graph.onn` seem to be the same as `Original_graph.onnx`

Dear dory developers,

I am using dory to generate my own dronet-network, which is an integer-quantized onnx file generated by nemo (master branch). However, while dory could deal with dory_examples like dronet_complete very well, it can not process my own network. I keep receiving the following issue:
DORY Frontend Check. Node Add is not accepted inside the DORY Frontend IR..
If I look inside the log folder, I found that the automatically generated05_DORY_Frontend_final_graph.onnx is the same as Original_graph.onnx.
Any insights on the possible reasons of this problem? Thank you for your time!

Cannot recursively clone dory

There are private git submodules in dory which makes it impossible for others to recursively clone the repo. Could you please move these private modules outside dory? This would be very convenient.

Dory support for Bias add in linear layer?

Hi,
I pushed this simple NN through the Nemo quantization process with no problem/

and the Nemo output this onnx file:

However, when I tried to push this through the Dory process, it has an error:
DORY Frontend Check. Node Add is not accepted inside the DORY Frontend IR.
caused by the bias add after the matmul layer. I check the code and it is this line:

dory/dory/Parsers/DORY_node.py

Line 124 in 4f0d2a3

if self.name == 'Addition' and len(self.input_indexes) == 1:

that prevented the Add layer becomes a "Addition" layer.

The other issue #41 mentioned the pattern might not be supported but the linear layer is said to be supported in the README. Could you help me identify the issue here and I am wondering if I could make a small change to make this work.

I am working on a publication for a top-tier journal and will cite these works in my work. Thank you so much for the help!

pulp-platform / dory Goto Github PK

dory's Issues

Recommend Projects

Recommend Topics

Recommend Org