traveller59 / torch2trt Goto Github PK

View Code? Open in Web Editor NEW

89.0 89.0 19.0 180 KB

convert torch module to tensorrt network or tvm function

License: MIT License

Python 100.00%

pytorch tensorrt tvm

torch2trt's Issues

Problem about the future of the project

@traveller59 Hey, the project is meaningful. It will update continuously ?

"aten::flatten" is missing

aten::flatten is NOT implemented as a handler, even though it is required when running benchmak.py

failed to inference

environment:
tensorrt:6.0.1.5 cuda:10.0 cudnn:7.6.0
i use the serialized resnet50.onnx which from the torchvision.models.resnet50.

eror log
[TensorRT] WARNING: TensorRT was linked against cuDNN 7.6.3 but loaded cuDNN 7.6.0 [TensorRT] WARNING: TensorRT was linked against cuDNN 7.6.3 but loaded cuDNN 7.6.0 [TensorRT] ERROR: ../rtSafe/cuda/cudaPoolingRunner.cpp (84) - Cudnn Error in execute: 8 (CUDNN_STATUS_EXECUTION_FAILED) [TensorRT] ERROR: FAILED_EXECUTION: std::exception
can u give me some advice? thanks

missing handler aten::group_norm

While converting pytorch model into trt I faced error. Please guide me how to fix it.

Exception has occurred: AssertionError
missing handler aten::group_norm, available handlers: ['prim::Cons....

can you have this code docker-image?

Hi, @traveller59 , I will have two questions:

1、this torch2trt project could transfer the second model to tensorrt(especially pointpillars) successfully ?

2、if so, can you have this project docker images ?

Thanks.

Inference is failed after loading the engine file

Hi,

Here is my environment setting:

CentOS 7.0
PyTorch 1.1.0
TensorRT 5.1.2.2 with CUDA9.0
CUDA9.0
cuDNN7.5.0 with CUDA9.0
Python3.6.8

After installed the dependencies of this repo.
I tried the test.py, and serialized the engine into file "test.engine"

Then I try to load it and execute the inference as normal tensorrt python code, as below:

def get_engine(engine_file):
        with open(engine_file,"rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
                print("Engine Loaded")
                return runtime.deserialize_cuda_engine(f.read())
        return None

if __name__ == '__main__':
    img = np.random.rand(1, 3,299,299)
    img /= 255.0
    img -= 0.5
    img *= 2.0
    bindings = []
    img = np.ascontiguousarray(img)                                                                                                                                
    engine = get_engine('test.engine')                                                                                                                            
    stream = cuda.Stream()
    context = engine.create_execution_context()
    output = np.empty(1000, dtype = np.float32)
    d_input = cuda.mem_alloc(1 * img.nbytes)
    d_output = cuda.mem_alloc(1 * output.nbytes)
    bindings = [int(d_input), int(d_output)]
    cuda.memcpy_htod_async(d_input, img, stream)
    context.execute_async(1, bindings, stream.handle, None)
    cuda.memcpy_dtoh_async(output, d_output, stream)
    stream.synchronize()

Then I get below error:

$ python3 test_run.py
Engine Loaded
[TensorRT] WARNING: TensorRT was compiled against cuDNN 7.5.0 but is linked against cuDNN 7.5.1. This mismatch may potentially cause undefined behavior.
[TensorRT] WARNING: TensorRT was compiled against cuDNN 7.5.0 but is linked against cuDNN 7.5.1. This mismatch may potentially cause undefined behavior.
[TensorRT] ERROR: Parameter check failed at: engine.cpp::enqueue::451, condition: bindings[x] != nullptr

Any advice will be welcome.
Thanks,

error with basic example

test with basic example code, an error occured:

Traceback (most recent call last):
  File "trt_test.py", line 27, in <module>
    trt_mode_out = graph_pth(img, verbose=True)
  File "/home/aisimba/.local/lib/python3.6/site-packages/torch2trt-1.0.0-py3.6.egg/torch2trt/core.py", line 665, in __call__
  File "/home/aisimba/.local/lib/python3.6/site-packages/torch2trt-1.0.0-py3.6.egg/torch2trt/core.py", line 497, in resolve_graph
  File "/home/aisimba/.local/lib/python3.6/site-packages/torch2trt-1.0.0-py3.6.egg/torch2trt/core.py", line 492, in resolve_graph
  File "/home/aisimba/.local/lib/python3.6/site-packages/torch2trt-1.0.0-py3.6.egg/torch2trt/handlers/constant.py", line 46, in prim_int
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

How can I convert my own model？

model = alexnet(pretrained=True).eval().cuda()
model = '/path/to/mymodel' or？

tensorRT error

when using trt.Builder(TRT_LOGGER) as builder, an error came out:

[TensorRT] ERROR: Cuda initialization failure with error 1. Please check cuda installation:  http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html.
Traceback (most recent call last):
  File "torch2trt.py", line 4, in <module>
    import torch2trt
  File "/home/aisimba/Documents/second.pytorch-v1.6/scripts/torch2trt.py", line 12, in <module>
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as trt_net:
TypeError: pybind11::init(): factory function returned nullptr

import torch
import torchvision
import tensorrt as trt 
import torch2trt
TRT_LOGGER = trt.Logger(trt.Logger.INFO)
# trt.Logger(trt.Logger.INFO)

# net = torchvision.models.inception_v3(pretrained=True).eval()
net = torchvision.models.squeezenet1_1(pretrained=True).eval()
inputs = torch.rand(1, 3, 416, 416)

with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as trt_net:
    pass

OS: Ubuntu 16.04
Python3.6
CUDA 10.1
TensorRT 5.1.2-1

traveller59 / torch2trt Goto Github PK

torch2trt's Issues

Problem about the future of the project

"aten::flatten" is missing

failed to inference

missing handler aten::group_norm

can you have this code docker-image?

Inference is failed after loading the engine file

error with basic example

How can I convert my own model？

tensorRT error

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent