Comments (7)
Bert-Pytorch demo is updated here. https://github.com/Tencent/Forward/tree/master/demo/bert
from forward.
@yuanzexi Bug when running test_torch_bert.py
("bert-base-uncased" downloaded from https://huggingface.co/bert-base-cased
),There are three errors messages.
[ERROR] 2021-04-08 16:22:09,532 trt_logger.cpp(64): [TRT] 29 [Cast]: could not find any supported formats consistent with input/output data types
[ERROR] 2021-04-08 16:22:09,533 trt_logger.cpp(64): [TRT] ../builder/cudnnBuilderGraphNodes.cpp (872) - Misc Error in reportPluginError: 0 (could not find any supported formats consistent with input/output data types)
[ERROR] 2021-04-08 16:22:09,550 trt_logger.cpp(64): [TRT] ../builder/cudnnBuilderGraphNodes.cpp (872) - Misc Error in reportPluginError: 0 (could not find any supported formats consistent with input/output data types)
[ERROR] 2021-04-08 16:22:09,550 trt_fwd_builder.cpp(121): buildEngineWithConfig error
[ERROR] 2021-04-08 16:52:12,292 torch_engine.cpp(219): Build forward engine failed
Traceback (most recent call last):
File "test_torch_bert.py", line 42, in <module>
TestForward(jit_path)
File "test_torch_bert.py", line 29, in TestForward
engine.save(engine_path)
AttributeError: 'NoneType' object has no attribute 'save'
Environment
TensorRT Version: 7.2.1.6
NVIDIA GPU: GTX1080TI
NVIDIA Driver Version: 450.80.02
CUDA Version: 11.0
CUDNN Version: 8.0.4
Operating System: centos 7.5
Python Version (if applicable): 3.6.13
PyTorch Version (if applicable): 1.7.1
transformers: 4.0.0
from forward.
Torch BERT models are only supported for Torch 1.3.1
API. Because BERT models exported by Torch 1.7.1
are different from that exported by Torch 1.3.1
. You could use the demo on Torch 1.3.1
. Thanks for your feedback. We will update soon for supporting Torch 1.7.1
BERT.
from forward.
@Fan9 The problem has been resolved in the newest master branch. You could try it again.
from forward.
I have successfully run BERT-Torch on torch=1.3.1
and torch=1.7.1
. Thanks for your help. Nice work!
from forward.
hi, @yuanzexi ,Have you tested the efficiency improvement of tensort? In my experiment, the speed of bert-trt
is greatly affected by the parameters of batch_size
and max_length
. In my code, builder.set_mode('float32'), bert-model:=BertForSequenceClassification
:
batch_size=1, max_length=52
,Bert-trt
is three times faster thanBert-pytorch
.batch_size=1, max_length=108
, double times faster.batch_size=32, max_length=52/108
,it takes the same time asBert-pytorch
.
Is there any problem here?
Environment
TensorRT Version: 7.2.1.6
NVIDIA GPU: GTX1080TI
NVIDIA Driver Version: 450.80.02
CUDA Version: 11.0
CUDNN Version: 8.0.4
Operating System: centos 7.5
Python Version (if applicable): 3.6.13
PyTorch Version (if applicable): 1.7.1
transformers: 4.0.0
from forward.
@Fan9 Thanks for your feedback. We had tested the performance of models in tensorrt, but we didn't try these configuration and environments. We'll try to profile this BERT model to figure out this problem.
from forward.
Related Issues (20)
- 使用这个Forward框架进行推断的时候会有精度损失吗? HOT 6
- 关于fwd-torch的几个路径的问题 HOT 2
- Segmentation fault (core dumped) when transfer keras model to trt. HOT 4
- Help to support tensorflow slim "Flatten" pattern to Tensorrt. HOT 10
- 编译VC项目时出错 HOT 14
- vs2017怎么跑demo? HOT 8
- Is have docker with all dependencies? HOT 3
- Will forward framework support paddlepaddle in the future? HOT 4
- win_python_keras版本加载模型时报错 HOT 4
- 编译出错 找不到cublas_device库 HOT 7
- keras.layer里Embedding的trt实现 HOT 8
- Keras中Flatten层的支持 HOT 2
- 对‘fwd::TrtForwardEngine::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)’未定义的引用 HOT 9
- 是否考虑重新写一个模型序列化?? HOT 3
- 与trtorch等项目的优劣对比 HOT 3
- make报错 HOT 22
- reflectPad存在两个问题 HOT 9
- 编译fwd-pytroch时提示缺少头文件 HOT 1
- 优化模型时报错 HOT 2
- [TRT] (Unnamed Layer* 0) [Convolution]: at least 4 dimensions are required for input HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from forward.