Comments (13)
when I test the 'interpolate' plugin and run notebooks/image_segmentation/conversion.ipynb ,I got this error
my tensorrt version: TensorRT-5.1.5.0.Ubuntu-16.04.5.x86_64-gnu.cuda-9.0.cudnn7.5
python3.5
When I run :
python3 -m torch2trt.test --name=interpolate
I got nothing info.
@jaybdub
from torch2trt.
May you provide an example code sample of using the interpolate
? Thanks
from torch2trt.
Hi sonack,
Thanks for reaching out! If installed and imported correctly, interpolate should appear in the list of registered converters. Could you share the following information
-
Which torch / TensorRT version are you using?
-
What steps did you take to install torch2trt with plugins (ie. did you previously install without plugins?)
For a full example, the image segmentation notebook converts a model that contains interpolate layers.
https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/notebooks/image_segmentation/conversion.ipynb
However, in interpolate.py we also register several module tests that would probably be the easiest thing to try first. You can execute them by calling
python3 -m torch2trt.test --name=interpolate
This will print information like the throughput, latency, as well as max absolute error (compared to PyTorch).
Please let me know if you have any other questions!
Best,
John
from torch2trt.
Hi John,
Hi sonack,
Thanks for reaching out! If installed and imported correctly, interpolate should appear in the list of registered converters. Could you share the following information
- Which torch / TensorRT version are you using?
The used TensorRT install package:
TensorRT-5.1.5.0.Ubuntu-14.04.5.x86_64-gnu.cuda-10.0.cudnn7.5.tar.gz
- What steps did you take to install torch2trt with plugins (ie. did you previously install without plugins?)
In order to compile torch2trt without error, I have made some fix in build.py
Below is the diff info:
diff --git a/build.py b/build.py
index 68b103b..024166f 100644
--- a/build.py
+++ b/build.py
@@ -7,23 +7,32 @@ PLUGINS = [
'interpolate',
]
-BASE_FOLDER = 'torch2trt/converters'
+def get_torch_dir():
+ return '/home/xxx/bin/libtorch/libtorch'
+
+def get_cuda_dir(version = '10.0'):
+ return '/usr/local/cuda-{}'.format(version)
+
+BASE_FOLDER = 'torch2trt/converters'
+# -lnvinfer
NINJA_STR = Template(
"""
rule link
- command = g++ -shared -o $$out $$in -L$torch_dir/lib -L$cuda_dir/lib64 -lc10 -lc10_cuda -ltorch -lcudart -lcaffe2 -lcaffe2_gpu -lprotobuf -lprotobuf-lite -pthread -lpthread -lnvinfer
+ command = g++ -shared -o $$out $$in -L$torch_dir/lib -L$cuda_dir/lib64 -lc10 -lc10_cuda -ltorch -lcudart -lcaffe2 -lcaffe2_gpu -lprotobuf -lprotobuf-lite -pthread -lpthread -std=c++11
rule protoc
command = protoc $$in --cpp_out=. --python_out=.
rule cxx
- command = g++ -c -fPIC $$in -I$cuda_dir/include -I$torch_dir/include -I$torch_dir/include/torch/csrc/api/include -I.
+ command = g++ -c -fPIC $$in -I$cuda_dir/include -I$torch_dir/include -I$torch_dir/include/torch/csrc/api/include -I. -std=c++11
"""
).substitute({
- 'torch_dir': imp.find_module('torch')[1],
- 'cuda_dir': '/usr/local/cuda'
+ # 'torch_dir': imp.find_module('torch')[1],
+ 'torch_dir': get_torch_dir(),
+ # 'cuda_dir': '/usr/local/cuda'
+ 'cuda_dir': get_cuda_dir(),
})
PLUGIN_TEMPLATE = Template(
(END)
Besides, I have installed protobuf-3.9.0, and libtorch(libtorch-shared-with-deps-latest.zip
) which is below:
The full compilation and install log is below without any complain:
(pytorch_py36) > python setup.py install --plugins
[4/4] g++ -shared -o torch2trt/libtorch2trt.so interpolate.pb.o interpolate.o -L/home/xxx/bin/lib... -lc10_cuda -ltorch -lcudart -lcaffe2 -lcaffe2_gpu -lprotobuf -lprotobuf-lite -pthread -lpthread -std=c++11
running install
running bdist_egg
running egg_info
creating torch2trt.egg-info
writing torch2trt.egg-info/PKG-INFO
writing dependency_links to torch2trt.egg-info/dependency_links.txt
writing top-level names to torch2trt.egg-info/top_level.txt
writing manifest file 'torch2trt.egg-info/SOURCES.txt'
reading manifest file 'torch2trt.egg-info/SOURCES.txt'
writing manifest file 'torch2trt.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/torch2trt
copying torch2trt/module_test.py -> build/lib/torch2trt
copying torch2trt/__init__.py -> build/lib/torch2trt
copying torch2trt/torch2trt.py -> build/lib/torch2trt
copying torch2trt/test.py -> build/lib/torch2trt
creating build/lib/torch2trt/converters
copying torch2trt/converters/relu.py -> build/lib/torch2trt/converters
copying torch2trt/converters/transpose.py -> build/lib/torch2trt/converters
copying torch2trt/converters/__init__.py -> build/lib/torch2trt/converters
copying torch2trt/converters/adaptive_avg_pool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/cat.py -> build/lib/torch2trt/converters
copying torch2trt/converters/AdaptiveAvgPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ReLU.py -> build/lib/torch2trt/converters
copying torch2trt/converters/identity.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ConvTranspose2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Identity.py -> build/lib/torch2trt/converters
copying torch2trt/converters/LogSoftmax.py -> build/lib/torch2trt/converters
copying torch2trt/converters/MaxPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/add.py -> build/lib/torch2trt/converters
copying torch2trt/converters/AvgPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/view.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Linear.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Conv2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ReLU6.py -> build/lib/torch2trt/converters
copying torch2trt/converters/iadd.py -> build/lib/torch2trt/converters
copying torch2trt/converters/mean.py -> build/lib/torch2trt/converters
copying torch2trt/converters/relu6.py -> build/lib/torch2trt/converters
copying torch2trt/converters/BatchNorm2d.py -> build/lib/torch2trt/converters
creating build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/__init__.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/interpolate.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/interpolate_pb2.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/libtorch2trt.so -> build/lib/torch2trt
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/module_test.py -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/torch2trt.py -> build/bdist.linux-x86_64/egg/torch2trt
creating build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/relu.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/transpose.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/adaptive_avg_pool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/cat.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/AdaptiveAvgPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ReLU.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/identity.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ConvTranspose2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Identity.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/LogSoftmax.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/MaxPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
creating build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/interpolate.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/interpolate_pb2.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/add.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/AvgPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/view.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Linear.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Conv2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ReLU6.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/iadd.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/mean.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/relu6.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/BatchNorm2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/libtorch2trt.so -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/test.py -> build/bdist.linux-x86_64/egg/torch2trt
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/module_test.py to module_test.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/torch2trt.py to torch2trt.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/relu.py to relu.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/transpose.py to transpose.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/adaptive_avg_pool2d.py to adaptive_avg_pool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/cat.py to cat.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/AdaptiveAvgPool2d.py to AdaptiveAvgPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ReLU.py to ReLU.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/identity.py to identity.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ConvTranspose2d.py to ConvTranspose2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Identity.py to Identity.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/LogSoftmax.py to LogSoftmax.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/MaxPool2d.py to MaxPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/interpolate.py to interpolate.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/interpolate_pb2.py to interpolate_pb2.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/add.py to add.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/AvgPool2d.py to AvgPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/view.py to view.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Linear.py to Linear.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Conv2d.py to Conv2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ReLU6.py to ReLU6.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/iadd.py to iadd.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/mean.py to mean.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/relu6.py to relu6.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/BatchNorm2d.py to BatchNorm2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/test.py to test.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
torch2trt.__pycache__.__init__.cpython-36: module references __file__
creating dist
creating 'dist/torch2trt-0.0.0-py3.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing torch2trt-0.0.0-py3.6.egg
removing '/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg' (and everything under it)
creating /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg
Extracting torch2trt-0.0.0-py3.6.egg to /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages
torch2trt 0.0.0 is already the active version in easy-install.pth
Installed /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg
Processing dependencies for torch2trt==0.0.0
Finished processing dependencies for torch2trt==0.0.0
and after installation, the directory would be:
There is no wrong when running the alexnet demo:
import torch
from torch2trt import torch2trt
from torchvision.models.alexnet import alexnet
# create some regular pytorch model...
model = alexnet(pretrained=True).eval().cuda()
# create example data
x = torch.ones((1, 3, 224, 224)).cuda()
# convert to TensorRT feeding sample data as input
model_trt = torch2trt(model, [x])
y = model(x)
y_trt = model_trt(x)
# check the output against PyTorch
print(torch.max(torch.abs(y - y_trt)))
As for whether I have installed torch2trt without plugins before, I think I might installed the torch2trt without plugins at a very first try, but I had uninstalled it before installing the with-plugins one. Does it matters?
But when I run my resnet50-based segmentation model, it gives
/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/functional.py:2539: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
File "convert_trt.py", line 29, in <module>
model_trt = torch2trt(model, [x])
File "/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py", line 252, in torch2trt
outputs = module(*inputs)
File "/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/home/xxx/Deploy/ocr_rec_internal/server/python/models/fpn_resnet.py", line 239, in forward
p4 = self._upsample_add(p5, c4)
File "/home/xxx/Deploy/ocr_rec_internal/server/python/models/fpn_resnet.py", line 214, in _upsample_add
return F.interpolate(x, size=(H, W), mode='bilinear') + y
File "/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py", line 97, in wrapper
converter(ctx)
File "/home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/add.py", line 10, in convert_add
layer = ctx.network.add_elementwise(input_a._trt, input_b._trt, trt.ElementWiseOperation.SUM)
AttributeError: 'Tensor' object has no attribute '_trt'
One more word here, the model was trained using F.Upsample
instead of F.interpolate
, I substitute it with interpolate
when deploying.
Below is the network definition:
def conv3x3(in_planes, out_planes, stride=1):
"""3x3 convolution with padding"""
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
padding=1, bias=False)
class convolution(nn.Module):
def __init__(self, k, inp_dim, out_dim, stride=1, with_bn=True):
super(convolution, self).__init__()
pad = (k - 1) // 2
self.conv = nn.Conv2d(inp_dim, out_dim, (k, k), padding=(pad, pad), stride=(stride, stride), bias=not with_bn)
self.bn = nn.BatchNorm2d(out_dim) if with_bn else nn.Sequential()
self.relu = nn.ReLU(inplace=True)
def forward(self, x):
conv = self.conv(x)
bn = self.bn(conv)
relu = self.relu(bn)
return relu
def make_cnv_layer(inp_dim, out_dim):
return convolution(3, inp_dim, out_dim)
def make_kp_layer(cnv_dim, curr_dim, out_dim):
return nn.Sequential(
convolution(3, cnv_dim, curr_dim, with_bn=False),
nn.Conv2d(curr_dim, out_dim, (1, 1))
)
class BasicBlock(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(BasicBlock, self).__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(Bottleneck, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * 4)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ResNet(nn.Module):
def __init__(self, block, layers, scale=1):
self.inplanes = 48
super(ResNet, self).__init__()
self.conv1 = nn.Conv2d(3, 48, kernel_size=7, stride=2, padding=3,
bias=False)
self.bn1 = nn.BatchNorm2d(48)
self.relu1 = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 48, layers[0])
self.layer2 = self._make_layer(block, 64, layers[1], stride=2)
self.layer3 = self._make_layer(block, 128, layers[2], stride=2)
self.layer4 = self._make_layer(block, 256, layers[3], stride=2)
# self.avgpool = nn.AvgPool2d(7, stride=1)
# self.fc = nn.Linear(512 * block.expansion, num_classes)
# Top layer
self.toplayer = nn.Conv2d(1024, 128, kernel_size=1, stride=1, padding=0) # Reduce channels
self.toplayer_bn = nn.BatchNorm2d(128)
self.toplayer_relu = nn.ReLU(inplace=True)
# Smooth layers
self.smooth1 = nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1)
self.smooth1_bn = nn.BatchNorm2d(128)
self.smooth1_relu = nn.ReLU(inplace=True)
self.smooth2 = nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1)
self.smooth2_bn = nn.BatchNorm2d(128)
self.smooth2_relu = nn.ReLU(inplace=True)
self.smooth3 = nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1)
self.smooth3_bn = nn.BatchNorm2d(128)
self.smooth3_relu = nn.ReLU(inplace=True)
# Lateral layers
self.latlayer1 = nn.Conv2d(512, 128, kernel_size=1, stride=1, padding=0)
self.latlayer1_bn = nn.BatchNorm2d(128)
self.latlayer1_relu = nn.ReLU(inplace=True)
self.latlayer2 = nn.Conv2d(256, 128, kernel_size=1, stride=1, padding=0)
self.latlayer2_bn = nn.BatchNorm2d(128)
self.latlayer2_relu = nn.ReLU(inplace=True)
##layer1*blockexpansion
self.latlayer3 = nn.Conv2d(48*4, 128, kernel_size=1, stride=1, padding=0)
self.latlayer3_bn = nn.BatchNorm2d(128)
self.latlayer3_relu = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(512, 128, kernel_size=3, stride=1, padding=1)
self.bn2 = nn.BatchNorm2d(128)
self.relu2 = nn.ReLU(inplace=True)
self.conv_label = make_cnv_layer(128, 64)
self.conv_center = make_cnv_layer(128, 64)
self.conv_up = make_cnv_layer(128, 64)
self.conv_down = make_cnv_layer(128, 64)
self.conv_label_last = make_kp_layer(64, 16, 3)
self.heat_center = make_kp_layer(64, 16, 1)
self.heat_up = make_kp_layer(64, 16, 1)
self.heat_down = make_kp_layer(64, 16, 1)
self.scale = scale
for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def _make_layer(self, block, planes, blocks, stride=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))
return nn.Sequential(*layers)
def _upsample(self, x, y, scale=1):
_, _, H, W = y.size()
# return F.upsample(x, size=(H // scale, W // scale), mode='bilinear')
# print('interpolate')
return F.interpolate(x, size=(H // scale, W // scale), mode='bilinear')
def _upsample_add(self, x, y):
_, _, H, W = y.size()
# return F.upsample(x, size=(H, W), mode='bilinear') + y
# print('interpolate add')
return F.interpolate(x, size=(H, W), mode='bilinear') + y
def forward(self, x):
h = x
h = self.conv1(h)
h = self.bn1(h)
h = self.relu1(h)
h = self.maxpool(h)
h = self.layer1(h)
c2 = h
h = self.layer2(h)
c3 = h
h = self.layer3(h)
c4 = h
h = self.layer4(h)
c5 = h
# Top-down
p5 = self.toplayer(c5)
p5 = self.toplayer_relu(self.toplayer_bn(p5))
c4 = self.latlayer1(c4)
c4 = self.latlayer1_relu(self.latlayer1_bn(c4))
p4 = self._upsample_add(p5, c4)
p4 = self.smooth1(p4)
p4 = self.smooth1_relu(self.smooth1_bn(p4))
c3 = self.latlayer2(c3)
c3 = self.latlayer2_relu(self.latlayer2_bn(c3))
p3 = self._upsample_add(p4, c3)
p3 = self.smooth2(p3)
p3 = self.smooth2_relu(self.smooth2_bn(p3))
c2 = self.latlayer3(c2)
c2 = self.latlayer3_relu(self.latlayer3_bn(c2))
p2 = self._upsample_add(p3, c2)
p2 = self.smooth3(p2)
p2 = self.smooth3_relu(self.smooth3_bn(p2))
p3 = self._upsample(p3, p2)
p4 = self._upsample(p4, p2)
p5 = self._upsample(p5, p2)
out = torch.cat((p2, p3, p4, p5), 1)
out = self.conv2(out)
out = self.relu2(self.bn2(out))
out_label = self.conv_label(out)
out_label = self.conv_label_last(out_label)
out_label = self._upsample(out_label, x, scale=self.scale)
out_label = F.softmax(out_label, dim=1)
out_center = self.conv_center(out)
out_center = self.heat_center(out_center)
out_center = self._upsample(out_center, x, scale=self.scale)
out_up = self.conv_up(out)
out_up = self.heat_up(out_up)
out_up = self._upsample(out_up, x, scale=self.scale)
out_down = self.conv_down(out)
out_down = self.heat_down(out_down)
out_down = self._upsample(out_down, x, scale=self.scale)
# pdb.set_trace()
# return torch.cat([out_label, out_center, out_up, out_down], 1)
return out_label, out_center, out_up, out_down
def resnet18(**kwargs):
model = ResNet(BasicBlock, [2, 2, 2, 2], **kwargs)
return model
def resnet34(**kwargs):
model = ResNet(BasicBlock, [3, 4, 6, 3], **kwargs)
return model
# I used this
def resnet50(**kwargs):
model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
return model
For a full example, the image segmentation notebook converts a model that contains interpolate layers.
https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/notebooks/image_segmentation/conversion.ipynb
Thanks, I will dive into it.
However, in interpolate.py we also register several module tests that would probably be the easiest thing to try first. You can execute them by calling
python3 -m torch2trt.test --name=interpolateThis will print information like the throughput, latency, as well as max absolute error (compared to PyTorch).
Please let me know if you have any other questions!
Best,
John
I really want to use this awesome repository, thanks for your helping!!! If you have something unclear, please contact me, thanks!
Best,
sonack
from torch2trt.
Do you have any idea? Thanks!
@jaybdub
from torch2trt.
Hi All,
@sonack @binzh93 Sorry for the delay on getting back. It is unusual that the build would pass, yet the interpolate test cases would not run. If interpolate.py is imported, we should at least see the test cases fail. Could you try removing the try/except statement around where interpolate is included and reinstalling
https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/torch2trt/converters/__init__.py#L31
To see if /what error is thrown? This should probably be made more verbose.
@binzh93 Could you share which version of torchvision you're using? The repository is under active development, it's possible they've made a breaking change since I tested. Ie. view
was recently replaced by flatten
in many of the models, which required a small update to torch2trt to parse.
Thanks for your patience all.
Best,
John
from torch2trt.
Hi All,
@sonack @binzh93 Sorry for the delay on getting back. It is unusual that the build would pass, yet the interpolate test cases would not run. If interpolate.py is imported, we should at least see the test cases fail. Could you try removing the try/except statement around where interpolate is included and reinstalling
https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/torch2trt/converters/__init__.py#L31
To see if /what error is thrown? This should probably be made more verbose.
@binzh93 Could you share which version of torchvision you're using? The repository is under active development, it's possible they've made a breaking change since I tested. Ie.
view
was recently replaced byflatten
in many of the models, which required a small update to torch2trt to parse.Thanks for your patience all.
Best,
John
OK! Thanks very much! I have solved the old issue, but new problem appeared.
I pulled the latest repository, and commented the try .... except
statement of interpolate importing, and recompiled it, but nothing happened:
Compilation Log:
(pytorch_py36) > python setup.py install --plugins
[4/4] g++ -shared -o torch2trt/libtorch2trt.so i...uf -lprotobuf-lite -pthread -lpthread -std=c++11
running install
running bdist_egg
running egg_info
creating torch2trt.egg-info
writing torch2trt.egg-info/PKG-INFO
writing dependency_links to torch2trt.egg-info/dependency_links.txt
writing top-level names to torch2trt.egg-info/top_level.txt
writing manifest file 'torch2trt.egg-info/SOURCES.txt'
reading manifest file 'torch2trt.egg-info/SOURCES.txt'
writing manifest file 'torch2trt.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/torch2trt
copying torch2trt/module_test.py -> build/lib/torch2trt
copying torch2trt/__init__.py -> build/lib/torch2trt
copying torch2trt/torch2trt.py -> build/lib/torch2trt
copying torch2trt/test.py -> build/lib/torch2trt
creating build/lib/torch2trt/converters
copying torch2trt/converters/relu.py -> build/lib/torch2trt/converters
copying torch2trt/converters/transpose.py -> build/lib/torch2trt/converters
copying torch2trt/converters/__init__.py -> build/lib/torch2trt/converters
copying torch2trt/converters/adaptive_avg_pool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/pad.py -> build/lib/torch2trt/converters
copying torch2trt/converters/cat.py -> build/lib/torch2trt/converters
copying torch2trt/converters/AdaptiveAvgPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/softmax.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ReLU.py -> build/lib/torch2trt/converters
copying torch2trt/converters/identity.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ConvTranspose2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Identity.py -> build/lib/torch2trt/converters
copying torch2trt/converters/BatchNorm1d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/sigmoid.py -> build/lib/torch2trt/converters
copying torch2trt/converters/LogSoftmax.py -> build/lib/torch2trt/converters
copying torch2trt/converters/MaxPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/tanh.py -> build/lib/torch2trt/converters
copying torch2trt/converters/add.py -> build/lib/torch2trt/converters
copying torch2trt/converters/AvgPool2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/div.py -> build/lib/torch2trt/converters
copying torch2trt/converters/view.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Linear.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Conv2d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/mul.py -> build/lib/torch2trt/converters
copying torch2trt/converters/Conv1d.py -> build/lib/torch2trt/converters
copying torch2trt/converters/ReLU6.py -> build/lib/torch2trt/converters
copying torch2trt/converters/iadd.py -> build/lib/torch2trt/converters
copying torch2trt/converters/mean.py -> build/lib/torch2trt/converters
copying torch2trt/converters/relu6.py -> build/lib/torch2trt/converters
copying torch2trt/converters/BatchNorm2d.py -> build/lib/torch2trt/converters
creating build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/__init__.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/interpolate.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/converters/interpolate/interpolate_pb2.py -> build/lib/torch2trt/converters/interpolate
copying torch2trt/libtorch2trt.so -> build/lib/torch2trt
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/module_test.py -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/torch2trt.py -> build/bdist.linux-x86_64/egg/torch2trt
creating build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/relu.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/transpose.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/adaptive_avg_pool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/pad.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/cat.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/AdaptiveAvgPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/softmax.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ReLU.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/identity.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ConvTranspose2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Identity.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/BatchNorm1d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/sigmoid.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/LogSoftmax.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/MaxPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
creating build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/__init__.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/interpolate.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/interpolate/interpolate_pb2.py -> build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate
copying build/lib/torch2trt/converters/tanh.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/add.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/AvgPool2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/div.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/view.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Linear.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Conv2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/mul.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/Conv1d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/ReLU6.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/iadd.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/mean.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/relu6.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/converters/BatchNorm2d.py -> build/bdist.linux-x86_64/egg/torch2trt/converters
copying build/lib/torch2trt/libtorch2trt.so -> build/bdist.linux-x86_64/egg/torch2trt
copying build/lib/torch2trt/test.py -> build/bdist.linux-x86_64/egg/torch2trt
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/module_test.py to module_test.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/torch2trt.py to torch2trt.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/relu.py to relu.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/transpose.py to transpose.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/adaptive_avg_pool2d.py to adaptive_avg_pool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/pad.py to pad.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/cat.py to cat.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/AdaptiveAvgPool2d.py to AdaptiveAvgPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/softmax.py to softmax.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ReLU.py to ReLU.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/identity.py to identity.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ConvTranspose2d.py to ConvTranspose2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Identity.py to Identity.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/BatchNorm1d.py to BatchNorm1d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/sigmoid.py to sigmoid.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/LogSoftmax.py to LogSoftmax.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/MaxPool2d.py to MaxPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/interpolate.py to interpolate.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/interpolate/interpolate_pb2.py to interpolate_pb2.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/tanh.py to tanh.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/add.py to add.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/AvgPool2d.py to AvgPool2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/div.py to div.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/view.py to view.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Linear.py to Linear.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Conv2d.py to Conv2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/mul.py to mul.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/Conv1d.py to Conv1d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/ReLU6.py to ReLU6.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/iadd.py to iadd.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/mean.py to mean.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/relu6.py to relu6.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/converters/BatchNorm2d.py to BatchNorm2d.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/torch2trt/test.py to test.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch2trt.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
torch2trt.__pycache__.__init__.cpython-36: module references __file__
creating dist
creating 'dist/torch2trt-0.0.0-py3.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing torch2trt-0.0.0-py3.6.egg
creating /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg
Extracting torch2trt-0.0.0-py3.6.egg to /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages
Adding torch2trt 0.0.0 to easy-install.pth file
Installed /home/xxx/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg
Processing dependencies for torch2trt==0.0.0
Finished processing dependencies for torch2trt==0.0.0
But when I import torch2trt
in python, it complains
(pytorch_py36) > python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch2trt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/xxx/bin/torch2trt_clean_3/torch2trt/__init__.py", line 2, in <module>
from .converters import *
File "/home/xxx/bin/torch2trt_clean_3/torch2trt/converters/__init__.py", line 32, in <module>
from .interpolate import *
File "/home/xxx/bin/torch2trt_clean_3/torch2trt/converters/interpolate/__init__.py", line 1, in <module>
from .interpolate import *
File "/home/xxx/bin/torch2trt_clean_3/torch2trt/converters/interpolate/interpolate.py", line 5, in <module>
from .interpolate_pb2 import interpolate_Message
File "/home/xxx/bin/torch2trt_clean_3/torch2trt/converters/interpolate/interpolate_pb2.py", line 7, in <module>
from google.protobuf import descriptor as _descriptor
ModuleNotFoundError: No module named 'google'
so I installed with pip install protobuf
, and recompile and reimport, it seems good:
However, when I test this installation on the (notebook)[https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/notebooks/image_segmentation/conversion.ipynb], it complains:
The full error log trace is
IndexError Traceback (most recent call last)
<ipython-input-9-7f1400d9157d> in <module>
----> 1 model_trt = torch2trt.torch2trt(model_w, [data], fp16_mode=True)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py in torch2trt(module, inputs, input_names, output_names, log_level, max_batch_size, fp16_mode, max_workspace_size, strict_type_constraints)
250 ctx.add_inputs(inputs, input_names)
251
--> 252 outputs = module(*inputs)
253
254 if not isinstance(outputs, tuple):
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
<ipython-input-6-8d085fb1ce64> in forward(self, x)
4 self.model = model
5 def forward(self, x):
----> 6 return self.model(x)['out']
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torchvision/models/segmentation/_utils.py in forward(self, x)
20 result = OrderedDict()
21 x = features["out"]
---> 22 x = self.classifier(x)
23 x = F.interpolate(x, size=input_shape, mode='bilinear', align_corners=False)
24 result["out"] = x
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/container.py in forward(self, input)
90 def forward(self, input):
91 for module in self._modules.values():
---> 92 input = module(input)
93 return input
94
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torchvision/models/segmentation/deeplabv3.py in forward(self, x)
89 res = []
90 for conv in self.convs:
---> 91 res.append(conv(x))
92 res = torch.cat(res, dim=1)
93 return self.project(res)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torchvision/models/segmentation/deeplabv3.py in forward(self, x)
59 size = x.shape[-2:]
60 x = super(ASPPPooling, self).forward(x)
---> 61 return F.interpolate(x, size=size, mode='bilinear', align_corners=False)
62
63
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py in wrapper(*args, **kwargs)
95
96 #print('%s : %s' % (method.__qualname__, converter.__name__))
---> 97 converter(ctx)
98
99 # convert to None so conversion will fail for unsupported layers
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/interpolate/interpolate.py in convert_interpolate(ctx)
32 size = list(output.shape[2:])
33
---> 34 plugin = get_interpolate_plugin(size=size, mode=mode, align_corners=align_corners)
35
36 layer = ctx.network.add_plugin_v2([input._trt], plugin)
~/anaconda2/envs/pytorch_py36/lib/python3.6/site-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/interpolate/interpolate.py in get_interpolate_plugin(size, mode, align_corners)
9 PLUGIN_NAME = 'interpolate'
10 registry = trt.get_plugin_registry()
---> 11 creator = [c for c in registry.plugin_creator_list if c.name == PLUGIN_NAME and c.plugin_namespace == 'torch2trt'][0]
12 message = interpolate_Message(size=size, mode=mode, align_corners=align_corners)
13 return creator.deserialize_plugin(PLUGIN_NAME, message.SerializeToString())
IndexError: list index out of range
The easiest way to reproduce this error is
How to solve it? Thank you very much!
from torch2trt.
@jaybdub FYI, this issue may be the same as #20
from torch2trt.
I forget to put my self-compiled protobuf before system's default protobuf library, which cause the error.
Solved. Closed.
from torch2trt.
protobuf
I met the same problem.
Which protobuf do you use?
from torch2trt.
protobuf
I met the same problem.
Which protobuf do you use?
I compiled the latest version on github 3.9.0
from torch2trt.
I sucessfully install interpolate plugin and import torch2trt with no bug, but when I use "python3 -m torch2trt.test --name=interpolate", "IndexError: list index out of range" error occured.
Do you sovle it ?Change the protobuf version?
My protobuf version is 3.0.0
torch version 1.1.0
torchvision version 0.3
@sonack
from torch2trt.
I sucessfully install interpolate plugin and import torch2trt with no bug, but when I use "python3 -m torch2trt.test --name=interpolate", "IndexError: list index out of range" error occured.
Do you sovle it ?Change the protobuf version?My protobuf version is 3.0.0
torch version 1.1.0
torchvision version 0.3
@sonack
My protobuf version is 3.9.0, one thing to note is to add -L${YOUR_PROTOBUF_LIB_PATH}
to rule link of build.py
, below is my build.py
and setup.py
, FYI.
build.py
import imp
import subprocess
import os
from string import Template
PLUGINS = [
'interpolate',
]
BASE_FOLDER = 'torch2trt/converters'
NINJA_TEMPLATE = Template((
"rule link\n"
" command = g++ -shared -o $$out $$in -L{{YOUR PROTOBUF INSTALL DIR}}/lib -L$torch_dir/lib -L$cuda_dir/lib64 -L$trt_lib_dir -lc10 -lc10_cuda -ltorch -lcudart -lcaffe2 -lcaffe2_gpu -lprotobuf -lprotobuf-lite -pthread -lpthread -lnvinfer -Xlinker --unresolved-symbols=ignore-in-shared-libs\n"
"rule protoc\n"
" command = protoc $$in --cpp_out=. --python_out=.\n"
"rule cxx\n"
" command = g++ -c -fPIC $$in -I$cuda_dir/include -I$torch_dir/include -I$torch_dir/include/torch/csrc/api/include -I. -std=c++11 -I$trt_inc_dir\n"
))
PLUGIN_TEMPLATE = Template((
"build $plugin_dir/$plugin.pb.h $plugin_dir/$plugin.pb.cc $plugin_dir/${plugin}_pb2.py: protoc $plugin_dir/$plugin.proto\n"
"build $plugin.pb.o: cxx $plugin_dir/$plugin.pb.cc\n"
"build $plugin.o: cxx $plugin_dir/$plugin.cpp\n"
))
def build(cuda_dir="/usr/local/cuda",
torch_dir=imp.find_module('torch')[1],
trt_inc_dir="/usr/include/aarch64-linux-gnu",
trt_lib_dir="/usr/lib/aarch64-linux-gnu"):
global PLUGINS, BASE_FOLDER, NINJA_TEMPLATE, PLUGIN_TEMPLATE
NINJA_STR = NINJA_TEMPLATE.substitute({
'torch_dir': torch_dir,
'cuda_dir': cuda_dir,
'trt_inc_dir': trt_inc_dir,
'trt_lib_dir': trt_lib_dir,
})
plugin_o_files = []
for plugin in PLUGINS:
NINJA_STR += \
PLUGIN_TEMPLATE.substitute({
'plugin': plugin,
'plugin_dir': os.path.join(BASE_FOLDER, plugin),
})
plugin_o_files += [plugin + '.pb.o', plugin + '.o']
NINJA_STR += Template((
"build torch2trt/libtorch2trt.so: link $o_files\n"
)).substitute({'o_files': ' '.join(plugin_o_files)})
with open('build.ninja', 'w') as f:
f.write(NINJA_STR)
subprocess.call(['ninja'])
if __name__ == '__main__':
build()
Note to replace {{}} with your own path!!!!
setup.py
import os
import glob
import shutil
from setuptools import setup, find_packages
from setuptools.command.install import install
from setuptools.command.develop import develop
from distutils.cmd import Command
from build import build
package_data = {}
plugins_user_options = [
('plugins', None, 'Build plugins'),
('cuda-dir=', None, 'Location of CUDA (if not default location)'),
('torch-dir=', None, 'Location of PyTorch (if not default location)'),
('trt-inc-dir=', None, 'Location of TensorRT include files (if not default location)'),
('trt-lib-dir=', None, 'Location of TensorRT libraries (if not default location)'),
]
def initialize_plugins_options(cmd_obj):
cmd_obj.plugins = False
cmd_obj.cuda_dir = None
cmd_obj.torch_dir = None
cmd_obj.trt_inc_dir = None
cmd_obj.trt_lib_dir = None
def run_plugins_compilation(cmd_obj):
if cmd_obj.plugins:
build_args = {}
if cmd_obj.cuda_dir:
build_args['cuda_dir'] = cmd_obj.cuda_dir
if cmd_obj.torch_dir:
build_args['torch_dir'] = cmd_obj.torch_dir
if cmd_obj.trt_inc_dir:
build_args['trt_inc_dir'] = cmd_obj.trt_inc_dir
if cmd_obj.trt_lib_dir:
build_args['trt_lib_dir'] = cmd_obj.trt_lib_dir
print('Building in plugin support')
build(**build_args)
package_data['torch2trt'] = ['libtorch2trt.so']
class DevelopCommand(develop):
description = "Builds the package and symlinks it into the PYTHONPATH"
user_options = develop.user_options + plugins_user_options
def initialize_options(self):
develop.initialize_options(self)
initialize_plugins_options(self)
def finalize_options(self):
develop.finalize_options(self)
def run(self):
run_plugins_compilation(self)
develop.run(self)
class InstallCommand(install):
description = "Builds the package"
user_options = install.user_options + plugins_user_options
def initialize_options(self):
install.initialize_options(self)
initialize_plugins_options(self)
def finalize_options(self):
install.finalize_options(self)
def run(self):
run_plugins_compilation(self)
install.run(self)
class CleanCommand(Command):
"""Custom clean command to tidy up the project root."""
PY_CLEAN_FILES = ['./build', './dist', './__pycache__', './*.pyc', './*.tgz', './*.egg-info']
description = "Command to tidy up the project root"
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
root_dir = os.path.dirname(os.path.realpath(__file__))
for path_spec in self.PY_CLEAN_FILES:
# Make paths absolute and relative to this path
abs_paths = glob.glob(os.path.normpath(os.path.join(root_dir, path_spec)))
for path in [str(p) for p in abs_paths]:
if not path.startswith(root_dir):
# Die if path in CLEAN_FILES is absolute + outside this directory
raise ValueError("%s is not a path inside %s" % (path, root_dir))
print('Removing %s' % os.path.relpath(path))
shutil.rmtree(path)
cmd_list = {
"Removing generated protobuf cc files": "find . -name '*.pb.cc' -print0 | xargs -0 rm -f;",
"Removing generated protobuf h files": "find . -name '*.pb.h' -print0 | xargs -0 rm -f;",
"Removing generated protobuf py files": "find . -name '*_pb2.py' -print0 | xargs -0 rm -f;",
"Removing generated ninja files": "find . -name '*.ninja*' -print0 | xargs -0 rm -f;",
"Removing generated o files": "find . -name '*.o' -print0 | xargs -0 rm -f;",
"Removing generated so files": "find . -name '*.so' -print0 | xargs -0 rm -f;",
}
for cmd, script in cmd_list.items():
print("{}".format(cmd))
os.system(script)
setup(
name='torch2trt',
version='0.0.0',
description='An easy to use PyTorch to TensorRT converter',
cmdclass={
'install': InstallCommand,
'clean': CleanCommand,
'develop': DevelopCommand,
},
packages=find_packages(),
package_data=package_data
)
#!/usr/bin/env bash
python setup.py install --plugins --cuda-dir=/usr/local/cuda-10.0 --trt-inc-dir=/data00/xxx/TensorRT/1404/TensorRT-5.1.5.0/include --trt-lib-dir=/data00/xxx/TensorRT/1404/TensorRT-5.1.5.0/lib
replace with your own path.
from torch2trt.
Related Issues (20)
- Inconsistent inference results between PyTorch and converted TensorRT model using with LogSoftmax operator
- Inconsistent inference results between PyTorch and converted TensorRT model using with Softplus operator
- Inconsistent inference results between PyTorch and converted TensorRT model using with Silu operator
- Inconsistent inference results between PyTorch and converted TensorRT model using with Normalize operator
- Inconsistent inference results between PyTorch and converted TensorRT model using with Linear operator
- Inconsistent inference results between PyTorch and converted TensorRT model with AvgPool2d operator
- Inconsistent inference results between PyTorch and converted TensorRT model with Pad operator
- repvggblock in torch2trt
- Inconsistent inference results between PyTorch and converted TensorRT model with Selu operator
- Inconsistent inference results between PyTorch and converted TensorRT model with MaxPool2d operator
- Inconsistent inference results between PyTorch and converted TensorRT model with BatchNorm or InstanceNorm operator
- Inconsistent inference results between PyTorch and converted TensorRT model with Interpolate operator
- Inconsistent inference results between PyTorch and converted TensorRT model with ConvTranspose2d operator
- 'tensorrt.tensorrt.Builder' object has no attribute 'build_cuda_engine'
- python model successfully tested on polygraphy tensorrt, but failed, when loading on torch2trt
- torch2trt shows error when converting a model, even though regular pytorch -> onnx -> tensorrt and inference is successful
- [Question & FeatureRequest] Is that possible to convert a torch.nn.Module to TRTMoudle? HOT 1
- Error in converting hybridnets model using torch2trt
- Package is not PEP 517 compliant, making it incompatible with package managing tools
- AttributeError: 'tensorrt_bindings.tensorrt.INetworkDefinition' object has no attribute 'add_convolution'. Did you mean: 'add_convolution_nd'? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torch2trt.