Comments (14)
chrishkchris
commented on July 4, 2024
maybe the version of protobuf used in conda package does not match, but I am not sure
from singa.
chrishkchris
commented on July 4, 2024
https://blog.csdn.net/kailvechao1536/article/details/103538689 this blog says that changes to protobuf 3.9 can solve problem for onnx
or this onnx issue
onnx/onnx#2434
from singa.
nudles
commented on July 4, 2024
which version of onnx are you using?
you may need to use the onnx that is compatible with SINGA.
Protobuf has many versions. It is a bit difficult to make other dependencies compatible with some protobuf versions.
from singa.
chrishkchris
commented on July 4, 2024
https://github.com/apache/singa/blob/dev/tool/conda/singa/meta.yaml
currently our conda metal.yaml is using
- libprotobuf 3.6.1
- onnx >=1.3.0
maybe the two are not compatible
from singa.
nudles
commented on July 4, 2024
you can compare the libprotobuf of SINGA and onnx via methods here conda/conda#2361
from singa.
chrishkchris
commented on July 4, 2024
OK, from the Travis CI build log the onnx version was onnx-1.6.0-py36he1b5a44_0
seems that protobuf is not specified (then should be the most latest version), so I may try to use later version.
dcsysh@panda8:~$ conda search onnx=1.6.0=py36he1b5a44_0 --info
Loading channels: done
onnx 1.6.0 py36he1b5a44_0
-------------------------
file name : onnx-1.6.0-py36he1b5a44_0.tar.bz2
name : onnx
version : 1.6.0
build : py36he1b5a44_0
build number: 0
size : 2.9 MB
license : MIT
subdir : linux-64
url : https://conda.anaconda.org/conda-forge/linux-64/onnx-1.6.0-py36he1b5a44_0.tar.bz2
md5 : 26073522c9dcaa50a8894c762668d7da
timestamp : 2019-09-28 06:49:12 UTC
constraints :
- python_abi * *_cp36m
dependencies:
- libgcc-ng >=7.3.0
- libstdcxx-ng >=7.3.0
- numpy
- protobuf
- python >=3.6,<3.7.0a0
- six
from singa.
chrishkchris
commented on July 4, 2024
Also, our singa dockerfile is using pip3 install protobuf, so I may get the same version to conda
Currently, the pip install protobuf version is 3.11.3, but our conda package uses 3.6.1, so it may be better to upgrade
(builder) dcsysh@panda8:~$ pip install protobuf
Collecting protobuf
Downloading protobuf-3.11.3-cp36-cp36m-manylinux1_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 4.0 MB/s
Requirement already satisfied: six>=1.9 in ./anaconda3/envs/builder/lib/python3.6/site-packages (from protobuf) (1.14.0)
Requirement already satisfied: setuptools in ./anaconda3/envs/builder/lib/python3.6/site-packages (from protobuf) (45.2.0.post20200210)
Installing collected packages: protobuf
Successfully installed protobuf-3.11.3
from singa.
chrishkchris
commented on July 4, 2024
So I will try the conda build singa onnx test case with protobuf 3.11.3 first
from singa.
nudles
commented on July 4, 2024
you can try. I got many troubles when I was trying to upgrade protobuf..
from singa.
joddiy
commented on July 4, 2024
Hi, all, please use onnx==1.5.0, we now support op set of version 10 for onnx 1.5.0.
https://github.com/onnx/onnx/blob/master/docs/Versioning.md#released-versions
from singa.
chrishkchris
commented on July 4, 2024
Hi, all, please use onnx==1.5.0, we now support op set of version 10 for onnx 1.5.0.
https://github.com/onnx/onnx/blob/master/docs/Versioning.md#released-versions
got it, I will fix the conda version of onnx to 1.5.0. Thanks a lot
from singa.
chrishkchris
commented on July 4, 2024
so in the dockerfile (everyone is using for development) we may need to set to onnx==1.5.0
from singa.
chrishkchris
commented on July 4, 2024
After quite a exhaustive search of conda package versions' combinations, I found that this seems to be working for both onnx and singa:
host:
- swig 3.0.12
- openblas 0.3.9
- protobuf 3.9.2
- glog 0.3.5
- numpy 1.11.3
- cudnn {{ cudnn }} # ['cudnn' in str(build_str)]
- dnnl {{ dnnl }}
- python {{ python }}
run:
- {{ pin_compatible('glog', max_pin='x.x') }}
- {{ pin_compatible('numpy', max_pin='x.x') }}
- {{ pin_compatible('dnnl', max_pin='x.x') }}
- cudnn {{ cudnn }} # ['cudnn' in str(build_str)]
- python {{ python }}
- libprotobuf 3.9.2
- libopenblas 0.3.9
- pillow
- future
- tqdm
- onnx 1.6.0
- deprecated 1.2.7
onnx 1.5 fails one more test case due to numerical error, so I choose onnx 1.6
However, now there are still two errors in the test case I am investigating why:
+ python run.py
..............................................................................................................................EE...................................................F............................................................................................................................................................................................................................
======================================================================
ERROR: test_clip (test_onnx_backend.TestPythonOnnxBackend)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 1896, in test_clip
name='test_clip_example')
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 55, in expect
outputs_dict = sonnx.run_node(onnx_node, input_tensors, opset_version)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/sonnx.py", line 1300, in run_node
return cls._run_node(onnx_node, inputs, handle, forward, opset_version)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/sonnx.py", line 1324, in _run_node
outputs = forward(*inputs) if handle is None else forward(
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/autograd.py", line 499, in clip
return Clip(min, max)(x)[0]
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/autograd.py", line 245, in __call__
return self._do_forward(*xs)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/autograd.py", line 295, in _do_forward
ys = self.forward(*xs)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/autograd.py", line 479, in forward
mask0 = singa.LTFloat(x, self.min)
TypeError: in method 'LTFloat', argument 2 of type 'float'
======================================================================
ERROR: test_clip_default (test_onnx_backend.TestPythonOnnxBackend)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 1946, in test_clip_default
name='test_clip_default_min')
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 55, in expect
outputs_dict = sonnx.run_node(onnx_node, input_tensors, opset_version)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/sonnx.py", line 1300, in run_node
return cls._run_node(onnx_node, inputs, handle, forward, opset_version)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/singa/sonnx.py", line 1324, in _run_node
outputs = forward(*inputs) if handle is None else forward(
TypeError: clip() missing 1 required positional argument: 'max'
======================================================================
FAIL: test_pow (test_onnx_backend.TestPythonOnnxBackend)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 1851, in test_pow
expect(node, inputs=[x, y], outputs=[z], name='test_pow_example')
File "/root/miniconda/conda-bld/singa_1584677090067/test_tmp/test/python/test_onnx_backend.py", line 59, in expect
decimal=5)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/numpy/testing/utils.py", line 918, in assert_array_almost_equal
precision=decimal)
File "/root/miniconda/conda-bld/singa_1584677090067/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/numpy/testing/utils.py", line 739, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 5 decimals
(mismatch 33.33333333333333%)
x: array([ 1., 32., 729.], dtype=float32)
y: array([ 1. , 32. , 729.00006], dtype=float32)
----------------------------------------------------------------------
Ran 400 tests in 1.563s
P.S. The above remaining two test case error on test_clip (onnx backend @joddiy) occurs also in docker container enivornment without conda, so needed to be fixed.
from singa.
chrishkchris
commented on July 4, 2024
In conclusion, I have posted an conda package versions' combinations that works for SINGA and onnx in conda environment, so I close this issue. The change of conda package version is addressed in #624
from singa.
Related Issues (20)
- Switch between CPU and GPU devices for cnn example HOT 4
- Save the downloaded datasets to local directory HOT 2
- Add running scripts for cnn and cifar_distributed_cnn examples HOT 4
- Intermediate information printing HOT 3
- Adding arguments for weight decay and momentum HOT 2
- Increase max epoch for cnn example for better convergence HOT 2
- Update CMakeLists.txt for release 4.0.0 HOT 1
- Check Apache license header for release 4.0.0
- OpenCL Compilation Fails
- Upload Release 4.0.0 Package to SVN HOT 1
- Update the NOTICE file for images HOT 1
- gitignore and gitmodules should be removed from the release tar file HOT 2
- Create a new branch dev-postgresql HOT 2
- update CMakeLists.txt HOT 2
- Update the version and compilation commands HOT 3
- update the runtime Dockerfile HOT 2
- update conda and docker files HOT 3
- AttributeError: module 'singa.singa_wrap' has no attribute 'Communicator' HOT 2
- Update bloodmnist example by refining inline comments HOT 2
- Update documentation for distributed training HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from singa.