Generate plain C++ code for inference of ONNX models without dependencies
This project was made as an alternative to a final exam for the assignment "Computer Organization II". You can read the writeup in docs/TP Final onnx2code.pdf (in Spanish).
The following models have been tested and work as expected.
| Model | Size |
|---|---|
| mnist | 26 KB |
| Super_Resolution | 240 KB |
| squeezenet1.1 | 9 MB |
| emotion_ferplus | 34 MB |
| inception-v2 | 44 MB |
| resnet50-caffe2-v1 | 98 MB |
| VGG 16 and VGG 16-bn | 527 MB |
| VGG 19 and VGG 19-bn | 548 MB |
| VGG 19-caffe2 | 561 MB |
- Minimum ONNX opset version: 7
- Quantized models are not supported
Only float data type is supported.
| Operator | Attribute support |
|---|---|
| Add, Div, Mul, Sub | ✅ with broadcasting |
| Concat | ✅ with multiple inputs ✅ axis |
| Conv | ✅ bias ✅ stride ✅ padding (and auto_pad)❌ dilations ❌ depthwise (group != 1) |
| Sum | ✅ with multiple inputs ❌ with broadcasting |
| Relu, Tanh, Sigmoid, Clip | ✅ |
| Gemm | ✅ with bias ❌ transpose A ✅ tranpose B ❌ alpha != 1 ❌ beta != 1 |
| Identity | ✅ |
| MaxPool, AveragePool | ✅ stride ✅ padding (and auto_pad)❌ dilations ❌ storage_order != 0 ❌ count_include_pad != 0 |
| Softmax | ✅ stride ✅ axis |
| Transpose | ✅ perm |
We provide a ready to use Docker image:
docker run --rm -it -v $pwd/mnist.onnx:/app/input.onnx:ro -v $pwd/output:/app/output:rw mlomb/onnx2code:latest --variations=im2col,loop-tiling --checks=3The command above will generate C++ code for the mnist.onnx model in the output folder.
- gcc (required if checking models)
- Python 3.10
- pipenv
Clone and install dependencies with pipenv install.
To generate code from an ONNX model, run the following command inside a pipenv shell:
python -m onnx2code --variation=im2col,loop-tiling mnist.onnx output_folder --checks=3
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent