run the distilled student with 8-bit quantization (onnxruntime) about fastformers HOT 6 CLOSED

Hi @baiziyuandyufei. I guess you followed the installation procedure.
Can you check the directory path /home/username/.local/lib/python3.6/site-packages/onnxruntime/__init__.py and the installed location with pip are the same?

from fastformers.

@ykim362 Thank you!
I used conda env enviroment, so I install the onnxruntime shouldn't use the --user parameter,
so I uninstall onnxruntime and install it by the command pip install examples/fastformers/onnxruntime/onnxruntime-1.4.0-cp36-cp36m-linux_x86_64.whl,

run the command
python examples/fastformers/run_superglue.py --model_type bert --model_name_or_path ../model/fastformers/student_model/student-4L-312 --task_name BoolQ --output_dir ./out --do_eval --data_dir ../dataset/fastformers/BoolQ --per_instance_eval_batch_size 1 --do_lower_case --max_seq_length 512 --use_onnxrt --no_cuda

but there is also have the error message:

Traceback (most recent call last):
  File "examples/fastformers/run_superglue.py", line 1901, in <module>
    main()
  File "examples/fastformers/run_superglue.py", line 1840, in main
    from onnxruntime import ExecutionMode, InferenceSession, SessionOptions
  File "/home/username/anaconda3/envs/fastformers/lib/python3.6/site-packages/onnxruntime/__init__.py", line 13, in <module>
    from onnxruntime.capi._pybind_state import get_all_providers, get_available_providers, get_device, set_seed, \
ImportError: cannot import name 'get_all_providers'

before the error message, there is a warning

/home/username/anaconda3/envs/fastformers/lib/python3.6/site-packages/onnxruntime/capi/_pybind_state.py:14: UserWarning: Cannot load onnxruntime.capi. Error: '/lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /home/username/anaconda3/envs/fastformers/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.so)'.

I install the glibc_2.27, user the command below

wget http://ftp.gnu.org/gnu/glibc/glibc-2.27.tar.gz
tar zxvf glibc-2.27.tar.gz
cd glibc-2.27
mkdir build
cd build
../configure --prefix=/opt/glibc-2.27
make -j4
sudo make install
export LD_LIBRARY_PATH="/opt/glibc-2.27/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

then, run the command
python examples/fastformers/run_superglue.py --model_type bert --model_name_or_path ../model/fastformers/student_model/student-4L-312 --task_name BoolQ --output_dir ./out --do_eval --data_dir ../dataset/fastformers/BoolQ --per_instance_eval_batch_size 1 --do_lower_case --max_seq_length 512 --use_onnxrt --no_cuda

get error :

segment fault

I guess the problem is the glibc version，so I change to the ubuntu18.04(replace the ubuntu16.04)，then it's OK.

from fastformers.

Is it running okay on Ubuntu 18.04? And, can you let me know your CPU model number?

from fastformers.

@ykim362 Yes, Ubuntu18.04 is OK.
model name : Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz

from fastformers.

@baiziyuandyufei Thanks for the confirmation!
It turns out that the binary has a dependency on glibc version. I am working on merging it into the main branch of onnxruntime. Once, it's done, I expect it supports broader environment.

from fastformers.

@ykim362 Thanks.

from fastformers.