- 🎯 喜欢python、transformers、nlp、pytorch
yuanzhoulvpi2017 / quick_sentence_transformers Goto Github PK
View Code? Open in Web Editor NEWsentence-transformers to onnx 让sbert模型推理效率更快
License: MIT License
sentence-transformers to onnx 让sbert模型推理效率更快
License: MIT License
这个批量跑数据会占用大量内存吗?比如类似sentence_transformers的batch-size的设置,来跑大量数据?
我有一个fine-tune之后的模型,包括了transformer、pooling以及dense三部分,这样子转换的话,只有transformer部分可以转换为onnx格式,剩下的pooling、dense部分还是不变,那我怎么在线上进行部署呢,只部署transformer部分的话,是不够的
测试下来,cpu下没有加速,反而比原始sentence-transformer要慢,这是什么原因呢
InferSentenceTransformer(model_name_or_path=model_path, device="cuda:1", onnx_model_name=onnx_model_name)
这里指定第二块卡时,运行时还是都用在第一块卡上
'shibing624/text2vec-base-chinese'是huggingface下载量超高的一个中文库,比sbert 的那个多语言库准确度高很多,看不到你说的两部分,sentence_transform 也支持,有办法加速吗
File "torch_to_onnx.py", line 190, in
conver_bert_torch_to_onnx()
File "torch_to_onnx.py", line 104, in conver_bert_torch_to_onnx
sess = onnxruntime.InferenceSession(MODEL_ONNX_PATH)
File "/opt/conda/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/opt/conda/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 379, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (Squeeze_1169) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 768
有遇到过这样的问题吗
第一次接触huggingface系列,按照项目的requirements安装了相关依赖后在使用_load_sbert_model.import_from_string读取模型文件时报错:
ImportError: cannot import name 'DatasetInfo' from 'huggingface_hub.hf_api' (C:\ProgramData\Anaconda3\lib\site-packages\huggingface_hub\hf_api.py)
看了看依赖里面确实缺少了这一个DatasetInfo,且在datasets中也没找到,请问这个如何解决?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.