在yolov7的循环代码中加速计时功能，发现每十次循环就会出现一次循环是200多ms 其它大多10ms以内，请问怎么回事？

I am trying to use the dynamic batch image processing feature in TensorRT-Alpha but am encountering an error. I have followed the instructions provided in the repository and have successfully exported the ONNX file, but when I try to run the image processing code I am getting an error. Can you please provide assistance in resolving this issue? Thank you.

When testing on Windows, it seems like there was a problem with the infer time, but I don't know where the problem came from

使用yolov8n訓練出的模型(class只有1)，使用python_with_dll測試部屬，推理準度與原本差非常多，想請問是否有哪裡需要隨著模型修改?

我下载了7月份yolov8官方模型，pt转onnx成功，但是onnx转trt失败；
我看了一下你博客的评论，也有人onnx转trt失败，说是版本要保持一致，我的软件版本都是和一样的，ultralytics==8.0.5，只有yolo的pt是最新的，所以是yolov8模型结构变了么？
那么我怎么样才能下载到ultralytics==8.0.5版本的yolo官方权重呢。

错误指向这个文件D:\code\vs2019\VC\Tools\MSVC\14.29.30133\include\memory

该如何解决这个error ,网上资料很少

使用yolov5推理时，只进行了同一类的NMS，那如何对不同的类也进行NMS勒

Hi, I am trying to export ONNX from the python source code provided in the TensorRT-Alpha project following the instructions, but I am encountering an error. The error message is as follows:

[ERROR] Unable to export model to ONNX format: <error message>

I have tried troubleshooting the issue by going through the instructions again and making sure all dependencies are installed, but the error persists. Any help would be greatly appreciated.

Steps to reproduce:

• Clone the TensorRT-Alpha repository
• Follow the instructions provided to export ONNX from the python source code
• Encounter the error above

Environment:

OS: Windows 10
Framework: PyTorch
TensorRT version: 7.1.3
CUDA version: 10.2
Python version: 3.7

Thanks!

global void decode_yolov8_device_kernel 函数最后添加下添加

for (size_t i = 0; i < 51; i++){
*pout_item++ = *(pitem+1+i);
}
是否正确？

bool YOLOV8::init(const std::vector& trtFile,int gpu_id =0)
{
// 1. init engine & context
if (trtFile.empty())
{
return false;
}

this->set_gpu(gpu_id);

cudaError_t cudaError = cudaSetDevice(gpu_id);
if (cudaError == cudaSuccess)
{
	std::cout << "GPU device selected successfully!" << std::endl;
}
else
{
	std::cout << "GPU device selection failed!" << std::endl;
	return false;
}

///=========================
[04/15/2023-23:32:04] [E] [TRT] 1: [convolutionRunner.cpp::nvinfer1::rt::task::CaskConvolutionRunner::execute::213] Error Code 1: Cask (Cask convolution execution)
[04/15/2023-23:32:04] [E] [TRT] 1: [checkMacros.cpp::nvinfer1::catchCudaError::278] Error Code 1: Cuda Runtime (an illegal memory access was encountered)

我按照视频操作编译的时候报了这个错

yolov8.exe --model=yolov8n.trt --size=640 --batch_size=1 --video=http://10.100.55.191:8081/video --show

我设置了这样的输入参数，他会在下面这处返回。然后我把他注释掉，就可以了 。

我这种简单粗暴的方法是不是有问题，麻烦能增加直播源的检测吗 。

问题成功解决，感谢飞哥大力援助，以下是，解决方法：
TX2 系统版本，jetpak4.6
关键步骤：
1.在PC or TX2导出静态onnx.
2.在TX2上，用TRT8.2编译onnx，得到trt文件.
注意：
然后编译tensorrt-alpha代码时所用的Tensorrt版本，要与trt转换时的一致。
关键命令：
1.在PC 或者 TX2导出静态onnx，注意这里与其他X86 ubuntu 上的转换命令不一致：
yolo mode=export model=yolov8n.pt format=onnx batch=1
2.将onnx文件拷贝到，TX2上，并在TX2上运行以下命令编译trt文件：
../../TensorRT-8.2.1.8/bin/trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.trt --buildOnly
TX2的trtexec可执行文件目录在：
/usr/src/tensorrt/bin
注意自行更改命令的目录
3.运行测试：
./app_yolov8 --model=../../data/yolov8/yolov8n.trt --size=640 --batch_size=1 --img=../../data/6406407.jpg --show

我尝试在TX2设备上部署YOLOV8项目，但是TX2上的tensorrt，转换模型时会报告不支持int64的错误，有什么方法可以转换为int32呢？
或者正确转换出模型呢？

`nvidia@ubuntu:~/TensorRT-Alpha-main/data/yolov8$ ./trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.trt --buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # ./trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.trt --buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
[02/28/2023-01:55:01] [I] === Model Options ===
[02/28/2023-01:55:01] [I] Format: ONNX
[02/28/2023-01:55:01] [I] Model: yolov8n.onnx
[02/28/2023-01:55:01] [I] Output:
[02/28/2023-01:55:01] [I] === Build Options ===
[02/28/2023-01:55:01] [I] Max batch: explicit batch
[02/28/2023-01:55:01] [I] Workspace: 16 MiB
[02/28/2023-01:55:01] [I] minTiming: 1
[02/28/2023-01:55:01] [I] avgTiming: 8
[02/28/2023-01:55:01] [I] Precision: FP32
[02/28/2023-01:55:01] [I] Calibration:
[02/28/2023-01:55:01] [I] Refit: Disabled
[02/28/2023-01:55:01] [I] Sparsity: Disabled
[02/28/2023-01:55:01] [I] Safe mode: Disabled
[02/28/2023-01:55:01] [I] DirectIO mode: Disabled
[02/28/2023-01:55:01] [I] Restricted mode: Disabled
[02/28/2023-01:55:01] [I] Save engine: yolov8n.trt
[02/28/2023-01:55:01] [I] Load engine:
[02/28/2023-01:55:01] [I] Profiling verbosity: 0
[02/28/2023-01:55:01] [I] Tactic sources: Using default tactic sources
[02/28/2023-01:55:01] [I] timingCacheMode: local
[02/28/2023-01:55:01] [I] timingCacheFile:
[02/28/2023-01:55:01] [I] Input(s)s format: fp32:CHW
[02/28/2023-01:55:01] [I] Output(s)s format: fp32:CHW
[02/28/2023-01:55:01] [I] Input build shape: images=1x3x640x640+4x3x640x640+8x3x640x640
[02/28/2023-01:55:01] [I] Input calibration shapes: model
[02/28/2023-01:55:01] [I] === System Options ===
[02/28/2023-01:55:01] [I] Device: 0
[02/28/2023-01:55:01] [I] DLACore:
[02/28/2023-01:55:01] [I] Plugins:
[02/28/2023-01:55:01] [I] === Inference Options ===
[02/28/2023-01:55:01] [I] Batch: Explicit
[02/28/2023-01:55:01] [I] Input inference shape: images=4x3x640x640
[02/28/2023-01:55:01] [I] Iterations: 10
[02/28/2023-01:55:01] [I] Duration: 3s (+ 200ms warm up)
[02/28/2023-01:55:01] [I] Sleep time: 0ms
[02/28/2023-01:55:01] [I] Idle time: 0ms
[02/28/2023-01:55:01] [I] Streams: 1
[02/28/2023-01:55:01] [I] ExposeDMA: Disabled
[02/28/2023-01:55:01] [I] Data transfers: Enabled
[02/28/2023-01:55:01] [I] Spin-wait: Disabled
[02/28/2023-01:55:01] [I] Multithreading: Disabled
[02/28/2023-01:55:01] [I] CUDA Graph: Disabled
[02/28/2023-01:55:01] [I] Separate profiling: Disabled
[02/28/2023-01:55:01] [I] Time Deserialize: Disabled
[02/28/2023-01:55:01] [I] Time Refit: Disabled
[02/28/2023-01:55:01] [I] Skip inference: Enabled
[02/28/2023-01:55:01] [I] Inputs:
[02/28/2023-01:55:01] [I] === Reporting Options ===
[02/28/2023-01:55:01] [I] Verbose: Disabled
[02/28/2023-01:55:01] [I] Averages: 10 inferences
[02/28/2023-01:55:01] [I] Percentile: 99
[02/28/2023-01:55:01] [I] Dump refittable layers:Disabled
[02/28/2023-01:55:01] [I] Dump output: Disabled
[02/28/2023-01:55:01] [I] Profile: Disabled
[02/28/2023-01:55:01] [I] Export timing to JSON file:
[02/28/2023-01:55:01] [I] Export output to JSON file:
[02/28/2023-01:55:01] [I] Export profile to JSON file:
[02/28/2023-01:55:01] [I]
[02/28/2023-01:55:01] [I] === Device Information ===
[02/28/2023-01:55:01] [I] Selected Device: NVIDIA Tegra X2
[02/28/2023-01:55:01] [I] Compute Capability: 6.2
[02/28/2023-01:55:01] [I] SMs: 2
[02/28/2023-01:55:01] [I] Compute Clock Rate: 1.3 GHz
[02/28/2023-01:55:01] [I] Device Global Memory: 7850 MiB
[02/28/2023-01:55:01] [I] Shared Memory per SM: 64 KiB
[02/28/2023-01:55:01] [I] Memory Bus Width: 128 bits (ECC disabled)
[02/28/2023-01:55:01] [I] Memory Clock Rate: 1.3 GHz
[02/28/2023-01:55:01] [I]
[02/28/2023-01:55:01] [I] TensorRT version: 8.2.1
[02/28/2023-01:55:03] [I] [TRT] [MemUsageChange] Init CUDA: CPU +266, GPU +0, now: CPU 285, GPU 6703 (MiB)
[02/28/2023-01:55:03] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 285 MiB, GPU 6704 MiB
[02/28/2023-01:55:03] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 314 MiB, GPU 6732 MiB
[02/28/2023-01:55:03] [I] Start parsing network model
[02/28/2023-01:55:03] [I] [TRT] ----------------------------------------------------------------
[02/28/2023-01:55:03] [I] [TRT] Input filename: yolov8n.onnx
[02/28/2023-01:55:03] [I] [TRT] ONNX IR version: 0.0.8
[02/28/2023-01:55:03] [I] [TRT] Opset version: 17
[02/28/2023-01:55:03] [I] [TRT] Producer name: pytorch
[02/28/2023-01:55:03] [I] [TRT] Producer version: 1.13.1
[02/28/2023-01:55:03] [I] [TRT] Domain:
[02/28/2023-01:55:03] [I] [TRT] Model version: 0
[02/28/2023-01:55:03] [I] [TRT] Doc string:
[02/28/2023-01:55:03] [I] [TRT] ----------------------------------------------------------------
[02/28/2023-01:55:03] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[02/28/2023-01:55:04] [E] [TRT] ModelImporter.cpp:773: While parsing node number 239 [Range -> "/model.22/Range_output_0"]:
[02/28/2023-01:55:04] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[02/28/2023-01:55:04] [E] [TRT] ModelImporter.cpp:775: input: "/model.22/Constant_8_output_0"
input: "/model.22/Cast_output_0"
input: "/model.22/Constant_9_output_0"
output: "/model.22/Range_output_0"
name: "/model.22/Range"
op_type: "Range"

[02/28/2023-01:55:04] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[02/28/2023-01:55:04] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:3352 In function importRange:
[8] Assertion failed: inputs.at(0).isInt32() && "For range operator with dynamic inputs, this version of TensorRT only supports INT32!"
[02/28/2023-01:55:04] [E] Failed to parse onnx file
[02/28/2023-01:55:04] [I] Finish parsing network model
[02/28/2023-01:55:04] [E] Parsing model failed
[02/28/2023-01:55:04] [E] Failed to create engine from model.
[02/28/2023-01:55:04] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # ./trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.trt --buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
nvidia@ubuntu:~/TensorRT-Alpha-main/data/yolov8$
`

我还尝试了这个方法，但是没什么作用，依然有int64的节点：
https://blog.csdn.net/dou3516/article/details/124577344

ubuntu20.04
TensorRT-8.6.1.6
OpenCV 4.6.0

[100%] Linking CXX executable app_yolov8
/usr/bin/ld: libyolov8.so: undefined reference to bgr2rgbDevice(int const&, float*, int, int, float*, int, int)' /usr/bin/ld: libyolov8.so: undefined reference to sample::splitToStringVec(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, char)'
/usr/bin/ld: libyolov8.so: undefined reference to nmsDeviceV1(utils::InitParameter, float*, int, int, int)' /usr/bin/ld: libyolov8.so: undefined reference to hwc2chwDevice(int const&, float*, int, int, float*, int, int)'
/usr/bin/ld: libyolov8.so: undefined reference to decodeDevice(utils::InitParameter, float*, int, int, int, float*, int, int)' /usr/bin/ld: libyolov8.so: undefined reference to __check_cuda_runtime(cudaError, char const*, char const*, int)'
/usr/bin/ld: libyolov8.so: undefined reference to yolov8::transposeDevice(utils::InitParameter, float*, int, int, int, float*, int, int)' /usr/bin/ld: libyolov8.so: undefined reference to nmsDeviceV2(utils::InitParameter, float*, int, int, int, int*, float*)'
/usr/bin/ld: libyolov8.so: undefined reference to yolov8::decodeDevice(utils::InitParameter, float*, int, int, int, float*, int, int)' /usr/bin/ld: libyolov8.so: undefined reference to resizeDevice(int const&, unsigned char*, int, int, float*, int, int, float, utils::AffineMat)'
/usr/bin/ld: libyolov8.so: undefined reference to `normDevice(int const&, float*, int, int, float*, int, int, utils::InitParameter)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/app_yolov8.dir/build.make:111: app_yolov8] Error 1
make[1]: *** [CMakeFiles/Makefile2:78: CMakeFiles/app_yolov8.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

Everything's running fine until I make yolov8.

有人遇到这个问题吗，怎么解决的

您好，感谢大佬您的仓库，我已经成功在win10上跑起来了yolov8模型，有几个问题想请教下：
1、能否出一个硬件解码CUDA解码图像的教程，设置batch-size为8进行推理，CPU占用率很高，应该是因为opencv去获取图像软解码，gpu使用率有点低？
2、在具体的部署过程中，多路实时的视频鹿，一个线程接入一路视频，对应一个模型，那么batch-size就对应1咯，那这样好像加速不太明显？
3、我测试一个1分40秒的视频，虽然推理显示很快，但是log打印显示的时间长达3-4分钟，想问问这是为啥呢？因为图像存储耗时吗？
4、batch-size设置为1，进行测试，加速效果不明显，正常吗？

[ 90%] Building CXX object CMakeFiles/app_yolov8.dir/app_yolov8.cpp.o
[100%] Linking CXX executable app_yolov8
/usr/bin/ld: libyolov8.so: undefined reference to `sample::splitToStringVec(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, char)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/app_yolov8.dir/build.make:163：app_yolov8] 错误 1
make[1]: *** [CMakeFiles/Makefile2:111：CMakeFiles/app_yolov8.dir/all] 错误 2
make: *** [Makefile:91：all] 错误 2

使用顯卡3060，輸入圖片為640*640的numpy.ndarray

這是使用官方一條龍的，包含轉檔成engine、推理

這是編譯成dll後從python呼叫的，模型轉換、推理都是使用TensorRT-Alpha

兩者模型皆使用fp16
可以看出使用此版本慢於ultralytics的tensorrt
但引用ultralytics實在太臃腫
使用大佬的舒服多了
想請問大佬如果只需要得到物件座標，是否有哪裡可以優化，或是我原本就有哪裡可能沒做好?

The latest verson of YOLOv5 is v7.0

I noticed that git checkout 6.0 command is noted in YOLOv5 TensorRT README.md

It has been a few month since the v7.0 released, and due to some reasons, I have trained my model on the latest version YOLOv5 (v7.0).

I wonder whether there's a chance that this repo may add support for v7.0 of YOLOv5 ?

Any further information is helpful. Looking forward to your reply.

I am trying to use the YOLOv4 model provided in the TensorRT-Alpha repository, but I am encountering an error when trying to run the inference. Specifically, I am getting the following message: CUDA

FAILED: app_yolov8
: && /usr/bin/c++ -Wno-error=deprecated-declarations -Wno-deprecated-declarations -g CMakeFiles/app_yolov8.dir/app_yolov8.cpp.o -o app_yolov8 -Wl,-rpath,/home/hy-20/project/TensorRT-Alpha/yolov8/cmake-build-debug:/home/hy-20/TensorRT-8.5.3.1/lib:/usr/local/cuda-11.8/lib64:/home/hy-20/opencv-4.5.5/build/lib libyolov8.so /usr/local/cuda-11.8/lib64/libcudart_static.a -ldl /usr/lib/x86_64-linux-gnu/librt.so /usr/local/cuda-11.8/lib64/libcublas.so /usr/local/cuda-11.8/lib64/libnppc.so /usr/local/cuda-11.8/lib64/libnppig.so /usr/local/cuda-11.8/lib64/libnppidei.so /usr/local/cuda-11.8/lib64/libnppial.so /home/hy-20/TensorRT-8.5.3.1/lib/libnvinfer.so /home/hy-20/TensorRT-8.5.3.1/lib/libnvinfer_plugin.so /home/hy-20/TensorRT-8.5.3.1/lib/libnvonnxparser.so /home/hy-20/TensorRT-8.5.3.1/lib/libnvcaffe_parser.so /home/hy-20/opencv-4.5.5/build/lib/libopencv_stitching.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_aruco.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_barcode.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_bgsegm.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_bioinspired.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_ccalib.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_dnn_objdetect.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_dnn_superres.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_dpm.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_face.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_freetype.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_fuzzy.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_hfs.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_img_hash.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_intensity_transform.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_line_descriptor.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_mcc.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_quality.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_rapid.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_reg.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_rgbd.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_saliency.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_stereo.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_structured_light.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_superres.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_surface_matching.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_tracking.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_videostab.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_wechat_qrcode.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_xfeatures2d.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_xobjdetect.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_xphoto.so.4.5.5 -lpthread /home/hy-20/opencv-4.5.5/build/lib/libopencv_phase_unwrapping.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_optflow.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_highgui.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_datasets.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_plot.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_text.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_videoio.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_ml.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_shape.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_ximgproc.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_video.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_imgcodecs.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_objdetect.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_calib3d.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_dnn.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_features2d.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_flann.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_photo.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_imgproc.so.4.5.5 /home/hy-20/opencv-4.5.5/build/lib/libopencv_core.so.4.5.5 && :
/usr/bin/ld: libyolov8.so: undefined reference to `sample::splitToStringVec(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, char)'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

请问这是因为环境版本不匹配还是因为什么啊？

[08/31/2023-10:01:48] [I] [TRT] [MemUsageChange] Init CUDA: CPU +402, GPU +0, now: CPU 13343, GPU 1875 (MiB)
[08/31/2023-10:01:48] [I] [TRT] Loaded engine size: 35 MiB
[08/31/2023-10:01:49] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +942, GPU +318, now: CPU 14343, GPU 2229 (MiB)
[08/31/2023-10:01:49] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/31/2023-10:01:49] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 14343, GPU 2229 (MiB)
[08/31/2023-10:01:49] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/31/2023-10:01:49] [I] the engine's info:
[08/31/2023-10:01:49] [I] idx = 0, images: -1, 3, 640, 640,
[08/31/2023-10:01:49] [I] idx = 1, output: -1, 25200, 6,
[08/31/2023-10:01:49] [I] the context's info:
[08/31/2023-10:01:49] [I] idx = 0, images: 4, 3, 640, 640,
[08/31/2023-10:01:49] [I] idx = 1, output: 4, 25200, 6,
yolo.infer time: 2.4728 ms
preprocess time = 0.08464; infer time = 0.585472; postprocess time = 0.013616
yolo.infer time: 2.5009 ms
preprocess time = 0.107832; infer time = 0.577024; postprocess time = 0.015384
yolo.infer time: 2.3982 ms
preprocess time = 0.06684; infer time = 0.579424; postprocess time = 0.013288
yolo.infer time: 2.3378 ms
preprocess time = 0.067432; infer time = 0.565504; postprocess time = 0.013408
yolo.infer time: 2.3586 ms
preprocess time = 0.068008; infer time = 0.5696; postprocess time = 0.012768
yolo.infer time: 2.6393 ms
preprocess time = 0.107336; infer time = 0.620544; postprocess time = 0.012104
yolo.infer time: 2.3641 ms
preprocess time = 0.067728; infer time = 0.570368; postprocess time = 0.013344
yolo.infer time: 2.3669 ms
preprocess time = 0.068152; infer time = 0.571464; postprocess time = 0.014944
yolo.infer time: 2.7903 ms
preprocess time = 0.068512; infer time = 0.676096; postprocess time = 0.013768
yolo.infer time: 2.3662 ms
preprocess time = 0.067496; infer time = 0.571152; postprocess time = 0.01484
yolo.infer time: 2.3795 ms
preprocess time = 0.068056; infer time = 0.574008; postprocess time = 0.015896
yolo.infer time: 2.7939 ms
preprocess time = 0.067672; infer time = 0.679456; postprocess time = 0.013592
yolo.infer time: 2.3601 ms
preprocess time = 0.067152; infer time = 0.5696; postprocess time = 0.016872
yolo.infer time: 2.3695 ms
preprocess time = 0.068152; infer time = 0.569448; postprocess time = 0.02248
yolo.infer time: 2.5756 ms
preprocess time = 0.06784; infer time = 0.623104; postprocess time = 0.015872
yolo.infer time: 2.3744 ms
preprocess time = 0.085144; infer time = 0.571904; postprocess time = 0.015608
yolo.infer time: 2.3629 ms
preprocess time = 0.067712; infer time = 0.570368; postprocess time = 0.013776
yolo.infer time: 2.3778 ms
preprocess time = 0.084304; infer time = 0.562176; postprocess time = 0.014976
yolo.infer time: 2.3147 ms
preprocess time = 0.06764; infer time = 0.556032; postprocess time = 0.016824
yolo.infer time: 2.3074 ms
preprocess time = 0.067624; infer time = 0.557568; postprocess time = 0.0142
yolo.infer time: 2.3045 ms
preprocess time = 0.067048; infer time = 0.556544; postprocess time = 0.015256
yolo.infer time: 2.3109 ms
preprocess time = 0.068048; infer time = 0.5568; postprocess time = 0.015208
yolo.infer time: 2.3144 ms
preprocess time = 0.067376; infer time = 0.55808; postprocess time = 0.01584
yolo.infer time: 2.308 ms
preprocess time = 0.06784; infer time = 0.556288; postprocess time = 0.014664
yolo.infer time: 2.7572 ms
preprocess time = 0.067288; infer time = 0.66924; postprocess time = 0.014096
yolo.infer time: 2.3519 ms
preprocess time = 0.06804; infer time = 0.566784; postprocess time = 0.01412
yolo.infer time: 2.3381 ms
preprocess time = 0.067656; infer time = 0.562176; postprocess time = 0.021808
yolo.infer time: 2.3428 ms
preprocess time = 0.067616; infer time = 0.56576; postprocess time = 0.013528
yolo.infer time: 2.3097 ms
preprocess time = 0.084904; infer time = 0.556112; postprocess time = 0.014288
yolo.infer time: 2.3046 ms
preprocess time = 0.068672; infer time = 0.556288; postprocess time = 0.012776
yolo.infer time: 2.3109 ms
preprocess time = 0.067344; infer time = 0.557312; postprocess time = 0.015184
yolo.infer time: 2.7682 ms
preprocess time = 0.067696; infer time = 0.671232; postprocess time = 0.015648
yolo.infer time: 2.3114 ms
preprocess time = 0.06756; infer time = 0.5568; postprocess time = 0.01468
yolo.infer time: 2.3074 ms
preprocess time = 0.067672; infer time = 0.556032; postprocess time = 0.013568
yolo.infer time: 2.3272 ms
preprocess time = 0.067688; infer time = 0.560936; postprocess time = 0.015976
yolo.infer time: 2.3166 ms
preprocess time = 0.067984; infer time = 0.55808; postprocess time = 0.02284
yolo.infer time: 2.3044 ms
preprocess time = 0.066928; infer time = 0.556832; postprocess time = 0.013672
yolo.infer time: 2.3214 ms
preprocess time = 0.066832; infer time = 0.557824; postprocess time = 0.014392
yolo.infer time: 2.7731 ms
preprocess time = 0.067904; infer time = 0.672512; postprocess time = 0.02192
yolo.infer time: 2.316 ms
preprocess time = 0.067848; infer time = 0.556808; postprocess time = 0.022432
yolo.infer time: 2.3362 ms
preprocess time = 0.067472; infer time = 0.563712; postprocess time = 0.012952
yolo.infer time: 2.5368 ms
preprocess time = 0.0684; infer time = 0.609024; postprocess time = 0.013744
yolo.infer time: 2.3136 ms
preprocess time = 0.066976; infer time = 0.558592; postprocess time = 0.01384
yolo.infer time: 2.3066 ms
preprocess time = 0.067696; infer time = 0.5568; postprocess time = 0.015248
yolo.infer time: 2.3113 ms
preprocess time = 0.06728; infer time = 0.557568; postprocess time = 0.014792
yolo.infer time: 2.372 ms
preprocess time = 0.129736; infer time = 0.5546; postprocess time = 0.013456
yolo.infer time: 2.3207 ms
preprocess time = 0.067648; infer time = 0.559616; postprocess time = 0.018424
yolo.infer time: 2.3107 ms
preprocess time = 0.067696; infer time = 0.557312; postprocess time = 0.022352
yolo.infer time: 2.7928 ms
preprocess time = 0.06708; infer time = 0.679; postprocess time = 0.022816
yolo.infer time: 2.3094 ms
preprocess time = 0.067784; infer time = 0.557312; postprocess time = 0.013496
yolo.infer time: 2.3203 ms
preprocess time = 0.068184; infer time = 0.557592; postprocess time = 0.013016
yolo.infer time: 2.7647 ms
preprocess time = 0.067728; infer time = 0.67072; postprocess time = 0.01304
yolo.infer time: 2.3192 ms
preprocess time = 0.066608; infer time = 0.557568; postprocess time = 0.012408
yolo.infer time: 2.7672 ms
preprocess time = 0.067936; infer time = 0.669184; postprocess time = 0.013808
yolo.infer time: 2.3111 ms
preprocess time = 0.067064; infer time = 0.556824; postprocess time = 0.013256
yolo.infer time: 2.3092 ms
preprocess time = 0.067912; infer time = 0.556288; postprocess time = 0.0162
yolo.infer time: 2.7753 ms
preprocess time = 0.067328; infer time = 0.67328; postprocess time = 0.013936
yolo.infer time: 2.3158 ms
preprocess time = 0.067704; infer time = 0.55808; postprocess time = 0.014192
yolo.infer time: 2.3163 ms
preprocess time = 0.06764; infer time = 0.557568; postprocess time = 0.0136
yolo.infer time: 2.3663 ms
preprocess time = 0.067656; infer time = 0.55936; postprocess time = 0.015072
yolo.infer time: 2.3435 ms
preprocess time = 0.06764; infer time = 0.563544; postprocess time = 0.016736
yolo.infer time: 4.1214 ms
preprocess time = 0.067488; infer time = 0.9024; postprocess time = 0.080352
yolo.infer time: 2.3136 ms
preprocess time = 0.067992; infer time = 0.5568; postprocess time = 0.013176
yolo.infer time: 2.4657 ms
preprocess time = 0.068472; infer time = 0.594784; postprocess time = 0.015312
yolo.infer time: 2.3097 ms
preprocess time = 0.084; infer time = 0.5568; postprocess time = 0.016824
yolo.infer time: 2.416 ms
preprocess time = 0.067808; infer time = 0.56448; postprocess time = 0.012544
yolo.infer time: 2.8042 ms
preprocess time = 0.068232; infer time = 0.679472; postprocess time = 0.014688
yolo.infer time: 2.3086 ms
preprocess time = 0.067448; infer time = 0.5568; postprocess time = 0.01284
yolo.infer time: 2.3172 ms
preprocess time = 0.066968; infer time = 0.557824; postprocess time = 0.014728
yolo.infer time: 2.3414 ms
preprocess time = 0.067864; infer time = 0.564992; postprocess time = 0.01644
yolo.infer time: 2.3614 ms
preprocess time = 0.083904; infer time = 0.558864; postprocess time = 0.014784
yolo.infer time: 2.3232 ms
preprocess time = 0.067552; infer time = 0.560896; postprocess time = 0.013944
yolo.infer time: 2.3161 ms
preprocess time = 0.067656; infer time = 0.557568; postprocess time = 0.0174
yolo.infer time: 2.3085 ms
preprocess time = 0.084456; infer time = 0.557568; postprocess time = 0.015064
yolo.infer time: 2.3129 ms
preprocess time = 0.06772; infer time = 0.55784; postprocess time = 0.017032
yolo.infer time: 2.3127 ms
preprocess time = 0.06768; infer time = 0.557568; postprocess time = 0.015024
yolo.infer time: 2.7748 ms
preprocess time = 0.067384; infer time = 0.673368; postprocess time = 0.015432
yolo.infer time: 2.3397 ms
preprocess time = 0.068152; infer time = 0.563456; postprocess time = 0.014336
yolo.infer time: 2.3147 ms
preprocess time = 0.0674; infer time = 0.558168; postprocess time = 0.014872
yolo.infer time: 2.3112 ms
preprocess time = 0.068448; infer time = 0.557824; postprocess time = 0.015096
yolo.infer time: 2.379 ms
preprocess time = 0.124568; infer time = 0.5568; postprocess time = 0.014544
yolo.infer time: 2.3088 ms
preprocess time = 0.067304; infer time = 0.556288; postprocess time = 0.015032
yolo.infer time: 2.3298 ms
preprocess time = 0.067744; infer time = 0.560976; postprocess time = 0.015176
yolo.infer time: 2.7644 ms
preprocess time = 0.067864; infer time = 0.670208; postprocess time = 0.014664
yolo.infer time: 2.3133 ms
preprocess time = 0.067664; infer time = 0.55764; postprocess time = 0.015664
yolo.infer time: 2.3128 ms
preprocess time = 0.067872; infer time = 0.557568; postprocess time = 0.013472
yolo.infer time: 2.3145 ms
preprocess time = 0.067712; infer time = 0.558592; postprocess time = 0.015416
yolo.infer time: 2.7824 ms
preprocess time = 0.0678; infer time = 0.675328; postprocess time = 0.015064
yolo.infer time: 2.3274 ms
preprocess time = 0.067096; infer time = 0.56192; postprocess time = 0.014664
yolo.infer time: 2.371 ms
preprocess time = 0.067144; infer time = 0.5568; postprocess time = 0.014584
yolo.infer time: 2.308 ms
preprocess time = 0.06764; infer time = 0.5568; postprocess time = 0.014312
yolo.infer time: 2.3692 ms
preprocess time = 0.125008; infer time = 0.554496; postprocess time = 0.014472
yolo.infer time: 2.3148 ms
preprocess time = 0.06724; infer time = 0.558592; postprocess time = 0.01508
yolo.infer time: 2.3184 ms
preprocess time = 0.067616; infer time = 0.556032; postprocess time = 0.023912
yolo.infer time: 2.7837 ms
preprocess time = 0.067856; infer time = 0.674376; postprocess time = 0.027128
yolo.infer time: 2.3093 ms
preprocess time = 0.067816; infer time = 0.5568; postprocess time = 0.01576
yolo.infer time: 2.338 ms
preprocess time = 0.067552; infer time = 0.563456; postprocess time = 0.014496
yolo.infer time: 2.3369 ms
preprocess time = 0.06776; infer time = 0.56376; postprocess time = 0.0142
yolo.infer time: 2.3092 ms
preprocess time = 0.136912; infer time = 0.556352; postprocess time = 0.014944
yolo.infer time: 2.3142 ms
preprocess time = 0.068144; infer time = 0.5568; postprocess time = 0.014456
yolo.infer time: 2.3051 ms
preprocess time = 0.067472; infer time = 0.555008; postprocess time = 0.015568
yolo.infer time: 2.4034 ms
preprocess time = 0.10676; infer time = 0.562704; postprocess time = 0.01516
yolo.infer time: 2.3169 ms
preprocess time = 0.067816; infer time = 0.558592; postprocess time = 0.014024
yolo.infer time: 2.3109 ms
preprocess time = 0.067832; infer time = 0.557136; postprocess time = 0.014856
yolo.infer time: 2.7912 ms
preprocess time = 0.068224; infer time = 0.676904; postprocess time = 0.015576
yolo.infer time: 2.3187 ms
preprocess time = 0.067344; infer time = 0.557344; postprocess time = 0.015752
yolo.infer time: 2.8962 ms
preprocess time = 0.067904; infer time = 0.61184; postprocess time = 0.104336
yolo.infer time: 2.3467 ms
preprocess time = 0.067608; infer time = 0.56576; postprocess time = 0.015184
yolo.infer time: 2.3455 ms
preprocess time = 0.066912; infer time = 0.563256; postprocess time = 0.012648
yolo.infer time: 2.3135 ms
preprocess time = 0.128712; infer time = 0.557144; postprocess time = 0.014592
yolo.infer time: 2.3077 ms
preprocess time = 0.067488; infer time = 0.556112; postprocess time = 0.014256
yolo.infer time: 2.3262 ms
preprocess time = 0.06744; infer time = 0.557056; postprocess time = 0.014152
yolo.infer time: 2.7631 ms
preprocess time = 0.066856; infer time = 0.669792; postprocess time = 0.015752
yolo.infer time: 2.3443 ms
preprocess time = 0.067032; infer time = 0.564736; postprocess time = 0.014584
yolo.infer time: 2.3336 ms
preprocess time = 0.0682; infer time = 0.563208; postprocess time = 0.01236
yolo.infer time: 2.3065 ms
preprocess time = 0.067264; infer time = 0.556632; postprocess time = 0.01708
yolo.infer time: 2.3193 ms
preprocess time = 0.084704; infer time = 0.55812; postprocess time = 0.01288
yolo.infer time: 2.3517 ms
preprocess time = 0.066776; infer time = 0.567552; postprocess time = 0.012816
yolo.infer time: 2.339 ms
preprocess time = 0.068152; infer time = 0.56404; postprocess time = 0.013264
yolo.infer time: 2.7721 ms
preprocess time = 0.067856; infer time = 0.671232; postprocess time = 0.021816
yolo.infer time: 2.344 ms
preprocess time = 0.067312; infer time = 0.564832; postprocess time = 0.015528
yolo.infer time: 2.3139 ms
preprocess time = 0.068152; infer time = 0.5568; postprocess time = 0.013608
yolo.infer time: 2.3132 ms
preprocess time = 0.067624; infer time = 0.557416; postprocess time = 0.02432
yolo.infer time: 2.7591 ms
preprocess time = 0.067064; infer time = 0.669952; postprocess time = 0.014952
yolo.infer time: 2.3606 ms
preprocess time = 0.067456; infer time = 0.55936; postprocess time = 0.013224
yolo.infer time: 2.3023 ms
preprocess time = 0.06784; infer time = 0.55612; postprocess time = 0.013168
yolo.infer time: 2.76 ms
preprocess time = 0.067488; infer time = 0.67072; postprocess time = 0.021096
yolo.infer time: 2.3422 ms
preprocess time = 0.067832; infer time = 0.563456; postprocess time = 0.021984
yolo.infer time: 2.3373 ms
preprocess time = 0.06788; infer time = 0.563456; postprocess time = 0.012488
yolo.infer time: 2.7588 ms
preprocess time = 0.067392; infer time = 0.670024; postprocess time = 0.013848
yolo.infer time: 2.3118 ms
preprocess time = 0.067648; infer time = 0.55708; postprocess time = 0.014544
yolo.infer time: 2.3146 ms
preprocess time = 0.068184; infer time = 0.557312; postprocess time = 0.014856
yolo.infer time: 2.3041 ms
preprocess time = 0.0678; infer time = 0.555872; postprocess time = 0.021064
yolo.infer time: 2.592 ms
preprocess time = 0.11424; infer time = 0.606008; postprocess time = 0.02168
yolo.infer time: 2.3137 ms
preprocess time = 0.067568; infer time = 0.557312; postprocess time = 0.013832
yolo.infer time: 2.3573 ms
preprocess time = 0.067176; infer time = 0.558848; postprocess time = 0.023072
yolo.infer time: 2.4501 ms
preprocess time = 0.0672; infer time = 0.595712; postprocess time = 0.01324
yolo.infer time: 2.307 ms
preprocess time = 0.067488; infer time = 0.558336; postprocess time = 0.013536
yolo.infer time: 2.3039 ms
preprocess time = 0.067808; infer time = 0.557312; postprocess time = 0.014784
yolo.infer time: 2.3282 ms
preprocess time = 0.067232; infer time = 0.562952; postprocess time = 0.014496
yolo.infer time: 2.784 ms
preprocess time = 0.06716; infer time = 0.675624; postprocess time = 0.015136
yolo.infer time: 2.312 ms
preprocess time = 0.067896; infer time = 0.556032; postprocess time = 0.013032
yolo.infer time: 2.3062 ms
preprocess time = 0.0674; infer time = 0.556616; postprocess time = 0.014896
yolo.infer time: 2.3101 ms
preprocess time = 0.067656; infer time = 0.557824; postprocess time = 0.01836
yolo.infer time: 2.5783 ms
preprocess time = 0.113552; infer time = 0.603136; postprocess time = 0.01512
yolo.infer time: 2.3073 ms
preprocess time = 0.067904; infer time = 0.556032; postprocess time = 0.013648
yolo.infer time: 2.6016 ms
preprocess time = 0.066688; infer time = 0.6272; postprocess time = 0.01556
yolo.infer time: 2.3004 ms
preprocess time = 0.067688; infer time = 0.556048; postprocess time = 0.013312
yolo.infer time: 2.3114 ms
preprocess time = 0.06804; infer time = 0.55612; postprocess time = 0.01488
yolo.infer time: 2.7441 ms
preprocess time = 0.06744; infer time = 0.666368; postprocess time = 0.013392
yolo.infer time: 2.3154 ms
preprocess time = 0.066672; infer time = 0.559104; postprocess time = 0.015432
yolo.infer time: 2.498 ms
preprocess time = 0.06732; infer time = 0.6016; postprocess time = 0.051352
yolo.infer time: 2.3079 ms
preprocess time = 0.06744; infer time = 0.556288; postprocess time = 0.015584
yolo.infer time: 2.3095 ms
preprocess time = 0.068056; infer time = 0.557856; postprocess time = 0.013584
yolo.infer time: 2.3128 ms
preprocess time = 0.067168; infer time = 0.558344; postprocess time = 0.015016
yolo.infer time: 2.6284 ms
preprocess time = 0.114512; infer time = 0.60628; postprocess time = 0.016336
yolo.infer time: 2.3211 ms
preprocess time = 0.067568; infer time = 0.557672; postprocess time = 0.019896
yolo.infer time: 2.334 ms
preprocess time = 0.067928; infer time = 0.56272; postprocess time = 0.013136
yolo.infer time: 2.7565 ms
preprocess time = 0.067576; infer time = 0.669216; postprocess time = 0.013224
yolo.infer time: 2.3364 ms
preprocess time = 0.067264; infer time = 0.564056; postprocess time = 0.01852
yolo.infer time: 2.3486 ms
preprocess time = 0.067328; infer time = 0.557312; postprocess time = 0.012872
yolo.infer time: 2.3934 ms
preprocess time = 0.068064; infer time = 0.558096; postprocess time = 0.014608
yolo.infer time: 2.3198 ms
preprocess time = 0.067416; infer time = 0.558592; postprocess time = 0.017968
yolo.infer time: 2.3267 ms
preprocess time = 0.068128; infer time = 0.560952; postprocess time = 0.015488
yolo.infer time: 2.3037 ms
preprocess time = 0.113944; infer time = 0.55552; postprocess time = 0.015256
yolo.infer time: 2.3085 ms
preprocess time = 0.06792; infer time = 0.555856; postprocess time = 0.024208
yolo.infer time: 2.4148 ms
preprocess time = 0.068512; infer time = 0.581896; postprocess time = 0.014952
yolo.infer time: 2.3072 ms
preprocess time = 0.083984; infer time = 0.5568; postprocess time = 0.015872
yolo.infer time: 2.355 ms
preprocess time = 0.067736; infer time = 0.557328; postprocess time = 0.016136
yolo.infer time: 2.3074 ms
preprocess time = 0.067416; infer time = 0.557056; postprocess time = 0.016376
yolo.infer time: 2.3088 ms
preprocess time = 0.136368; infer time = 0.556336; postprocess time = 0.014456
yolo.infer time: 2.3391 ms
preprocess time = 0.067752; infer time = 0.563232; postprocess time = 0.014952
yolo.infer time: 2.3129 ms
preprocess time = 0.066504; infer time = 0.556032; postprocess time = 0.015216
yolo.infer time: 2.3733 ms
preprocess time = 0.12884; infer time = 0.555336; postprocess time = 0.016272
yolo.infer time: 2.3083 ms
preprocess time = 0.067648; infer time = 0.556544; postprocess time = 0.015256
yolo.infer time: 2.3211 ms
preprocess time = 0.06768; infer time = 0.55936; postprocess time = 0.015136
yolo.infer time: 2.3121 ms
preprocess time = 0.067776; infer time = 0.557312; postprocess time = 0.01536
yolo.infer time: 2.3063 ms
preprocess time = 0.06768; infer time = 0.557056; postprocess time = 0.019752
yolo.infer time: 2.3432 ms
preprocess time = 0.068064; infer time = 0.565504; postprocess time = 0.015464
yolo.infer time: 2.409 ms
preprocess time = 0.085256; infer time = 0.55792; postprocess time = 0.015784
yolo.infer time: 2.3088 ms
preprocess time = 0.067544; infer time = 0.557312; postprocess time = 0.0126
yolo.infer time: 2.3038 ms
preprocess time = 0.067432; infer time = 0.557072; postprocess time = 0.015432
yolo.infer time: 2.307 ms
preprocess time = 0.084568; infer time = 0.556288; postprocess time = 0.015128
yolo.infer time: 2.3046 ms
preprocess time = 0.067376; infer time = 0.557312; postprocess time = 0.015128
yolo.infer time: 2.3387 ms
preprocess time = 0.067352; infer time = 0.56448; postprocess time = 0.017192
yolo.infer time: 2.3757 ms
preprocess time = 0.133936; infer time = 0.556544; postprocess time = 0.02536
yolo.infer time: 2.3259 ms
preprocess time = 0.068376; infer time = 0.560648; postprocess time = 0.013784
yolo.infer time: 2.3056 ms
preprocess time = 0.067256; infer time = 0.5568; postprocess time = 0.0288
yolo.infer time: 2.6061 ms
preprocess time = 0.0668; infer time = 0.630528; postprocess time = 0.014648
yolo.infer time: 2.3209 ms
preprocess time = 0.067824; infer time = 0.559616; postprocess time = 0.024408
yolo.infer time: 2.3267 ms
preprocess time = 0.067496; infer time = 0.557568; postprocess time = 0.013944
yolo.infer time: 2.3122 ms
preprocess time = 0.084632; infer time = 0.556544; postprocess time = 0.016416
yolo.infer time: 2.308 ms
preprocess time = 0.067304; infer time = 0.555808; postprocess time = 0.013856
yolo.infer time: 2.3316 ms
preprocess time = 0.067888; infer time = 0.562176; postprocess time = 0.01552
yolo.infer time: 2.5762 ms
preprocess time = 0.114224; infer time = 0.594944; postprocess time = 0.013968
yolo.infer time: 2.307 ms
preprocess time = 0.06744; infer time = 0.556832; postprocess time = 0.045344
yolo.infer time: 2.3372 ms
preprocess time = 0.067496; infer time = 0.563456; postprocess time = 0.017184
yolo.infer time: 2.3295 ms
preprocess time = 0.067464; infer time = 0.562688; postprocess time = 0.014912
yolo.infer time: 2.3368 ms
preprocess time = 0.067208; infer time = 0.557352; postprocess time = 0.016536
yolo.infer time: 2.7575 ms
preprocess time = 0.067656; infer time = 0.668928; postprocess time = 0.013224
yolo.infer time: 2.3101 ms
preprocess time = 0.06744; infer time = 0.557824; postprocess time = 0.014488
yolo.infer time: 2.3069 ms
preprocess time = 0.067728; infer time = 0.55636; postprocess time = 0.026344
yolo.infer time: 2.3061 ms
preprocess time = 0.067464; infer time = 0.557312; postprocess time = 0.013672
yolo.infer time: 2.3094 ms
preprocess time = 0.06764; infer time = 0.5568; postprocess time = 0.01736
yolo.infer time: 2.3151 ms
preprocess time = 0.06772; infer time = 0.557096; postprocess time = 0.022832
yolo.infer time: 2.3379 ms
preprocess time = 0.067576; infer time = 0.563456; postprocess time = 0.016816
yolo.infer time: 2.3132 ms
preprocess time = 0.067752; infer time = 0.5576; postprocess time = 0.014584
yolo.infer time: 2.3166 ms
preprocess time = 0.067664; infer time = 0.557056; postprocess time = 0.022296
yolo.infer time: 2.3082 ms
preprocess time = 0.067928; infer time = 0.556544; postprocess time = 0.021632
yolo.infer time: 2.3312 ms
preprocess time = 0.084872; infer time = 0.560896; postprocess time = 0.01552
yolo.infer time: 2.3104 ms
preprocess time = 0.06772; infer time = 0.557312; postprocess time = 0.019272
yolo.infer time: 2.3453 ms
preprocess time = 0.068408; infer time = 0.564304; postprocess time = 0.01376
yolo.infer time: 2.3144 ms
preprocess time = 0.06728; infer time = 0.557112; postprocess time = 0.014168
yolo.infer time: 2.3174 ms
preprocess time = 0.067544; infer time = 0.558144; postprocess time = 0.013008
yolo.infer time: 2.3099 ms
preprocess time = 0.068144; infer time = 0.557312; postprocess time = 0.013176
yolo.infer time: 2.3668 ms
preprocess time = 0.084688; infer time = 0.570112; postprocess time = 0.013712
yolo.infer time: 2.3186 ms
preprocess time = 0.067656; infer time = 0.557368; postprocess time = 0.013504
yolo.infer time: 2.302 ms
preprocess time = 0.067192; infer time = 0.556288; postprocess time = 0.013392
yolo.infer time: 2.3241 ms
preprocess time = 0.083864; infer time = 0.560128; postprocess time = 0.013272
yolo.infer time: 2.3126 ms
preprocess time = 0.067368; infer time = 0.556856; postprocess time = 0.013912
yolo.infer time: 2.3079 ms
preprocess time = 0.067264; infer time = 0.555776; postprocess time = 0.013192
yolo.infer time: 2.7596 ms
preprocess time = 0.068504; infer time = 0.669456; postprocess time = 0.01448
yolo.infer time: 2.3051 ms
preprocess time = 0.067384; infer time = 0.556032; postprocess time = 0.02184
yolo.infer time: 2.3095 ms
preprocess time = 0.067344; infer time = 0.556032; postprocess time = 0.013712
yolo.infer time: 2.4791 ms
preprocess time = 0.06776; infer time = 0.598528; postprocess time = 0.015104
yolo.infer time: 2.3089 ms
preprocess time = 0.067416; infer time = 0.557056; postprocess time = 0.021656
yolo.infer time: 2.3134 ms
preprocess time = 0.06748; infer time = 0.558128; postprocess time = 0.020832
yolo.infer time: 2.3088 ms
preprocess time = 0.067176; infer time = 0.556288; postprocess time = 0.016384
yolo.infer time: 2.7536 ms
preprocess time = 0.067752; infer time = 0.668672; postprocess time = 0.014408
yolo.infer time: 2.3052 ms
preprocess time = 0.06796; infer time = 0.556136; postprocess time = 0.01712
yolo.infer time: 2.3477 ms
preprocess time = 0.06836; infer time = 0.564992; postprocess time = 0.023016
yolo.infer time: 2.3135 ms
preprocess time = 0.084088; infer time = 0.557312; postprocess time = 0.022144
yolo.infer time: 2.3505 ms
preprocess time = 0.066976; infer time = 0.561712; postprocess time = 0.022448
yolo.infer time: 2.7956 ms
preprocess time = 0.067344; infer time = 0.665856; postprocess time = 0.01312
yolo.infer time: 2.3132 ms
preprocess time = 0.067936; infer time = 0.556544; postprocess time = 0.021504
yolo.infer time: 2.313 ms
preprocess time = 0.067672; infer time = 0.557568; postprocess time = 0.013448
yolo.infer time: 2.3133 ms
preprocess time = 0.067232; infer time = 0.556544; postprocess time = 0.025184
yolo.infer time: 2.3163 ms
preprocess time = 0.06756; infer time = 0.558592; postprocess time = 0.013384
yolo.infer time: 2.3092 ms
preprocess time = 0.0682; infer time = 0.557056; postprocess time = 0.014688
yolo.infer time: 2.3774 ms
preprocess time = 0.123808; infer time = 0.55636; postprocess time = 0.013648
yolo.infer time: 2.3079 ms
preprocess time = 0.067088; infer time = 0.556288; postprocess time = 0.013568
yolo.infer time: 2.3292 ms
preprocess time = 0.067712; infer time = 0.557056; postprocess time = 0.015184
yolo.infer time: 2.316 ms
preprocess time = 0.068456; infer time = 0.557056; postprocess time = 0.026744
yolo.infer time: 2.7434 ms
preprocess time = 0.067624; infer time = 0.666368; postprocess time = 0.015176
yolo.infer time: 2.3038 ms
preprocess time = 0.06756; infer time = 0.556032; postprocess time = 0.013096
yolo.infer time: 2.3161 ms
preprocess time = 0.068232; infer time = 0.557824; postprocess time = 0.012264
yolo.infer time: 2.334 ms
preprocess time = 0.067464; infer time = 0.56304; postprocess time = 0.013416
yolo.infer time: 2.3349 ms
preprocess time = 0.085432; infer time = 0.558848; postprocess time = 0.01488
yolo.infer time: 2.3174 ms
preprocess time = 0.066568; infer time = 0.557336; postprocess time = 0.022304
yolo.infer time: 2.8661 ms
preprocess time = 0.066568; infer time = 0.69512; postprocess time = 0.022872
yolo.infer time: 4.1345 ms
preprocess time = 0.107576; infer time = 1.01377; postprocess time = 0.014408
yolo.infer time: 4.1424 ms
preprocess time = 0.10868; infer time = 1.01274; postprocess time = 0.015744
yolo.infer time: 4.1278 ms
preprocess time = 0.107536; infer time = 1.01094; postprocess time = 0.015216
yolo.infer time: 4.1103 ms
preprocess time = 0.107144; infer time = 1.0071; postprocess time = 0.015168
yolo.infer time: 4.2796 ms
preprocess time = 0.21872; infer time = 1.00582; postprocess time = 0.022136
yolo.infer time: 4.1384 ms
preprocess time = 0.107504; infer time = 1.0135; postprocess time = 0.016184
yolo.infer time: 4.8914 ms
preprocess time = 0.10792; infer time = 1.20096; postprocess time = 0.015656
yolo.infer time: 4.1102 ms
preprocess time = 0.107632; infer time = 1.00762; postprocess time = 0.016304
yolo.infer time: 5.1193 ms
preprocess time = 0.10844; infer time = 1.24723; postprocess time = 0.015928
yolo.infer time: 4.2555 ms
preprocess time = 0.110688; infer time = 1.04322; postprocess time = 0.016816
yolo.infer time: 4.2412 ms
preprocess time = 0.11184; infer time = 1.04013; postprocess time = 0.014472
yolo.infer time: 4.2294 ms
preprocess time = 0.141504; infer time = 1.0368; postprocess time = 0.01584
yolo.infer time: 4.2293 ms
preprocess time = 0.111272; infer time = 1.03654; postprocess time = 0.015552
yolo.infer time: 4.8621 ms
preprocess time = 0.11208; infer time = 1.19457; postprocess time = 0.015832
yolo.infer time: 4.2481 ms
preprocess time = 0.110992; infer time = 1.04166; postprocess time = 0.015504
yolo.infer time: 5.1622 ms
preprocess time = 0.111128; infer time = 1.27053; postprocess time = 0.016328
yolo.infer time: 4.2399 ms
preprocess time = 0.11112; infer time = 1.03936; postprocess time = 0.01596
yolo.infer time: 4.3439 ms
preprocess time = 0.116304; infer time = 1.06419; postprocess time = 0.018184
yolo.infer time: 5.3188 ms
preprocess time = 0.115792; infer time = 1.30824; postprocess time = 0.016768
yolo.infer time: 4.3454 ms
preprocess time = 0.115424; infer time = 1.0647; postprocess time = 0.014584
yolo.infer time: 4.3721 ms
preprocess time = 0.146592; infer time = 1.07008; postprocess time = 0.014968
yolo.infer time: 4.3678 ms
preprocess time = 0.115336; infer time = 1.07136; postprocess time = 0.02252
yolo.infer time: 5.1437 ms
preprocess time = 0.11592; infer time = 1.2649; postprocess time = 0.015688
yolo.infer time: 4.4219 ms
preprocess time = 0.117592; infer time = 1.06882; postprocess time = 0.01668
yolo.infer time: 4.8074 ms
preprocess time = 0.197464; infer time = 1.13024; postprocess time = 0.015288
yolo.infer time: 4.4835 ms
preprocess time = 0.119856; infer time = 1.09875; postprocess time = 0.014976
yolo.infer time: 5.3904 ms
preprocess time = 0.120128; infer time = 1.32736; postprocess time = 0.015192
yolo.infer time: 4.4774 ms
preprocess time = 0.120456; infer time = 1.0967; postprocess time = 0.025344
yolo.infer time: 4.5217 ms
preprocess time = 0.151328; infer time = 1.10982; postprocess time = 0.017064
yolo.infer time: 4.4677 ms
preprocess time = 0.168928; infer time = 1.09568; postprocess time = 0.01752
yolo.infer time: 4.6735 ms
preprocess time = 0.252768; infer time = 1.09952; postprocess time = 0.016544
yolo.infer time: 4.5049 ms
preprocess time = 0.176192; infer time = 1.10182; postprocess time = 0.016816
yolo.infer time: 4.5987 ms
preprocess time = 0.175648; infer time = 1.12462; postprocess time = 0.0148
yolo.infer time: 5.5689 ms
preprocess time = 0.174896; infer time = 1.37037; postprocess time = 0.016248
yolo.infer time: 4.6822 ms
preprocess time = 0.174272; infer time = 1.13638; postprocess time = 0.016672
yolo.infer time: 6.0188 ms
preprocess time = 0.181168; infer time = 1.4825; postprocess time = 0.017744
yolo.infer time: 4.6874 ms
preprocess time = 0.174144; infer time = 1.13843; postprocess time = 0.015624
yolo.infer time: 5.8722 ms
preprocess time = 0.173784; infer time = 1.44768; postprocess time = 0.015104
yolo.infer time: 4.6253 ms
preprocess time = 0.174384; infer time = 1.13562; postprocess time = 0.016272
yolo.infer time: 5.8455 ms
preprocess time = 0.18048; infer time = 1.43978; postprocess time = 0.014888
yolo.infer time: 4.6981 ms
preprocess time = 0.18008; infer time = 1.15277; postprocess time = 0.014808
yolo.infer time: 5.4139 ms
preprocess time = 0.178128; infer time = 1.33248; postprocess time = 0.01668
yolo.infer time: 4.7516 ms
preprocess time = 0.183144; infer time = 1.1671; postprocess time = 0.017048
yolo.infer time: 5.6673 ms
preprocess time = 0.182824; infer time = 1.39162; postprocess time = 0.015768
yolo.infer time: 4.7525 ms
preprocess time = 0.1784; infer time = 1.16659; postprocess time = 0.033968
yolo.infer time: 4.9151 ms
preprocess time = 0.338984; infer time = 1.15763; postprocess time = 0.016424
yolo.infer time: 4.7461 ms
preprocess time = 0.180632; infer time = 1.16557; postprocess time = 0.015296
yolo.infer time: 5.5323 ms
preprocess time = 0.188816; infer time = 1.3079; postprocess time = 0.015376
yolo.infer time: 4.8574 ms
preprocess time = 0.183584; infer time = 1.19347; postprocess time = 0.01552
yolo.infer time: 4.857 ms
preprocess time = 0.365728; infer time = 1.19194; postprocess time = 0.019616
yolo.infer time: 4.8378 ms
preprocess time = 0.188432; infer time = 1.18784; postprocess time = 0.017032
yolo.infer time: 4.8845 ms
preprocess time = 0.18228; infer time = 1.19938; postprocess time = 0.016744
yolo.infer time: 5.8968 ms
preprocess time = 0.181168; infer time = 1.45282; postprocess time = 0.01592
yolo.infer time: 4.8597 ms
preprocess time = 0.187256; infer time = 1.19322; postprocess time = 0.016496
yolo.infer time: 5.6132 ms
preprocess time = 0.186144; infer time = 1.38291; postprocess time = 0.01536
yolo.infer time: 4.8455 ms
preprocess time = 0.18172; infer time = 1.19066; postprocess time = 0.015376
yolo.infer time: 5.5566 ms
preprocess time = 0.187416; infer time = 1.36781; postprocess time = 0.015144
yolo.infer time: 4.956 ms
preprocess time = 0.192512; infer time = 1.21758; postprocess time = 0.017392
yolo.infer time: 5.5405 ms
preprocess time = 0.189912; infer time = 1.36448; postprocess time = 0.034264
yolo.infer time: 5.0508 ms
preprocess time = 0.186544; infer time = 1.22624; postprocess time = 0.015192
yolo.infer time: 5.8979 ms
preprocess time = 0.186184; infer time = 1.4528; postprocess time = 0.017216
yolo.infer time: 4.9871 ms
preprocess time = 0.192264; infer time = 1.22522; postprocess time = 0.016184
yolo.infer time: 4.969 ms
preprocess time = 0.275888; infer time = 1.21651; postprocess time = 0.017312
yolo.infer time: 5.023 ms
preprocess time = 0.186456; infer time = 1.23494; postprocess time = 0.016008
yolo.infer time: 5.0859 ms
preprocess time = 0.192656; infer time = 1.25005; postprocess time = 0.01632
yolo.infer time: 5.087 ms
preprocess time = 0.197624; infer time = 1.25056; postprocess time = 0.026432
yolo.infer time: 5.0878 ms
preprocess time = 0.266368; infer time = 1.2493; postprocess time = 0.018272
yolo.infer time: 5.1058 ms
preprocess time = 0.190424; infer time = 1.25542; postprocess time = 0.016336
yolo.infer time: 5.0805 ms
preprocess time = 0.176656; infer time = 1.24979; postprocess time = 0.015824
yolo.infer time: 5.2007 ms
preprocess time = 0.196032; infer time = 1.27872; postprocess time = 0.043616
yolo.infer time: 5.0826 ms
preprocess time = 0.191776; infer time = 1.25056; postprocess time = 0.017112
yolo.infer time: 5.2556 ms
preprocess time = 0.145848; infer time = 1.29203; postprocess time = 0.018224
yolo.infer time: 5.8746 ms
preprocess time = 0.372008; infer time = 1.39366; postprocess time = 0.017232
yolo.infer time: 5.2419 ms
preprocess time = 0.201784; infer time = 1.28896; postprocess time = 0.016984
yolo.infer time: 5.231 ms
preprocess time = 0.180704; infer time = 1.28668; postprocess time = 0.02032
yolo.infer time: 5.9659 ms
preprocess time = 0.197216; infer time = 1.46918; postprocess time = 0.017192
yolo.infer time: 5.3137 ms
preprocess time = 0.201064; infer time = 1.29664; postprocess time = 0.017496
yolo.infer time: 5.2408 ms
preprocess time = 0.197064; infer time = 1.28666; postprocess time = 0.021056
yolo.infer time: 6.3811 ms
preprocess time = 0.202352; infer time = 1.57338; postprocess time = 0.020824
yolo.infer time: 5.4082 ms
preprocess time = 0.207048; infer time = 1.32173; postprocess time = 0.019336
yolo.infer time: 5.4125 ms
preprocess time = 0.220936; infer time = 1.32173; postprocess time = 0.0212
yolo.infer time: 6.3951 ms
preprocess time = 0.20588; infer time = 1.49713; postprocess time = 0.0176
yolo.infer time: 5.381 ms
preprocess time = 0.201592; infer time = 1.31994; postprocess time = 0.017072
yolo.infer time: 5.3834 ms
preprocess time = 0.200512; infer time = 1.32352; postprocess time = 0.016848
yolo.infer time: 5.7836 ms
preprocess time = 0.207832; infer time = 1.39622; postprocess time = 0.019008
yolo.infer time: 5.37 ms
preprocess time = 0.200728; infer time = 1.32099; postprocess time = 0.01656
yolo.infer time: 6.865 ms
preprocess time = 0.203864; infer time = 1.6855; postprocess time = 0.017328
yolo.infer time: 5.4422 ms
preprocess time = 0.20824; infer time = 1.33786; postprocess time = 0.020952
yolo.infer time: 5.4307 ms
preprocess time = 0.203856; infer time = 1.33606; postprocess time = 0.01692
yolo.infer time: 6.7226 ms
preprocess time = 0.208584; infer time = 1.65837; postprocess time = 0.016568
yolo.infer time: 5.4232 ms
preprocess time = 0.208368; infer time = 1.33532; postprocess time = 0.015328
yolo.infer time: 5.4799 ms
preprocess time = 0.389824; infer time = 1.34042; postprocess time = 0.016768
yolo.infer time: 5.4562 ms
preprocess time = 0.208072; infer time = 1.34195; postprocess time = 0.017096
yolo.infer time: 5.4761 ms
preprocess time = 0.204896; infer time = 1.34835; postprocess time = 0.015904
yolo.infer time: 5.6331 ms
preprocess time = 0.209472; infer time = 1.38675; postprocess time = 0.019952
yolo.infer time: 5.6372 ms
preprocess time = 0.196848; infer time = 1.37933; postprocess time = 0.020016
yolo.infer time: 5.6152 ms
preprocess time = 0.209984; infer time = 1.3824; postprocess time = 0.016584
yolo.infer time: 5.7116 ms
preprocess time = 0.212536; infer time = 1.39648; postprocess time = 0.016672
yolo.infer time: 6.8893 ms
preprocess time = 0.209504; infer time = 1.70191; postprocess time = 0.0158
yolo.infer time: 5.6611 ms
preprocess time = 0.2168; infer time = 1.38752; postprocess time = 0.021896
yolo.infer time: 5.6258 ms
preprocess time = 0.20996; infer time = 1.38419; postprocess time = 0.018128
yolo.infer time: 6.9594 ms
preprocess time = 0.213432; infer time = 1.7175; postprocess time = 0.018224
yolo.infer time: 6.4375 ms
preprocess time = 0.214; infer time = 1.58771; postprocess time = 0.016912
yolo.infer time: 5.7183 ms
preprocess time = 0.212536; infer time = 1.40731; postprocess time = 0.017224
yolo.infer time: 6.9459 ms
preprocess time = 0.219184; infer time = 1.71443; postprocess time = 0.017376
yolo.infer time: 5.7207 ms
preprocess time = 0.213232; infer time = 1.40878; postprocess time = 0.017032
yolo.infer time: 6.7636 ms
preprocess time = 0.214024; infer time = 1.49197; postprocess time = 0.02148
yolo.infer time: 5.6845 ms
preprocess time = 0.212168; infer time = 1.40083; postprocess time = 0.017216
yolo.infer time: 5.7708 ms
preprocess time = 0.230888; infer time = 1.42208; postprocess time = 0.018136
yolo.infer time: 6.5144 ms
preprocess time = 0.216528; infer time = 1.60872; postprocess time = 0.016704
yolo.infer time: 5.838 ms
preprocess time = 0.215936; infer time = 1.43795; postprocess time = 0.016856
yolo.infer time: 7.1775 ms
preprocess time = 0.225616; infer time = 1.75974; postprocess time = 0.016296
yolo.infer time: 5.8046 ms
preprocess time = 0.216288; infer time = 1.42976; postprocess time = 0.01668
yolo.infer time: 5.8031 ms
preprocess time = 0.215776; infer time = 1.42899; postprocess time = 0.01524
yolo.infer time: 6.8303 ms
preprocess time = 0.232088; infer time = 1.68627; postprocess time = 0.015504
yolo.infer time: 5.882 ms
preprocess time = 0.22292; infer time = 1.44947; postprocess time = 0.0162
yolo.infer time: 6.0269 ms
preprocess time = 0.192184; infer time = 1.47354; postprocess time = 0.016072
yolo.infer time: 7.1413 ms
preprocess time = 0.228328; infer time = 1.76334; postprocess time = 0.016632
yolo.infer time: 5.9937 ms
preprocess time = 0.222392; infer time = 1.4769; postprocess time = 0.017328
yolo.infer time: 6.6544 ms
preprocess time = 0.429224; infer time = 1.57645; postprocess time = 0.016816
yolo.infer time: 6.5287 ms
preprocess time = 0.222624; infer time = 1.61075; postprocess time = 0.02056
yolo.infer time: 5.9906 ms
preprocess time = 0.223304; infer time = 1.47558; postprocess time = 0.017104
yolo.infer time: 5.9821 ms
preprocess time = 0.234272; infer time = 1.47482; postprocess time = 0.018792
yolo.infer time: 12.8156 ms
preprocess time = 0.289944; infer time = 3.17952; postprocess time = 0.034488
yolo.infer time: 14.2456 ms
preprocess time = 0.210944; infer time = 3.5395; postprocess time = 0.028792
yolo.infer time: 14.6901 ms
preprocess time = 0.298152; infer time = 3.6503; postprocess time = 0.029552
yolo.infer time: 13.4756 ms
preprocess time = 0.281384; infer time = 3.21024; postprocess time = 0.024712
yolo.infer time: 10.2988 ms
preprocess time = 0.24436; infer time = 2.55181; postprocess time = 0.0252
yolo.infer time: 10.4816 ms
preprocess time = 0.347824; infer time = 2.5961; postprocess time = 0.025416
yolo.infer time: 10.7073 ms
preprocess time = 0.280048; infer time = 2.65395; postprocess time = 0.025128
yolo.infer time: 10.7219 ms
preprocess time = 0.260776; infer time = 2.65856; postprocess time = 0.025032
yolo.infer time: 10.6279 ms
preprocess time = 0.2492; infer time = 2.63478; postprocess time = 0.029096
yolo.infer time: 10.6948 ms
preprocess time = 0.26236; infer time = 2.65216; postprocess time = 0.02556
yolo.infer time: 10.6578 ms
preprocess time = 0.432008; infer time = 2.64166; postprocess time = 0.025896
yolo.infer time: 10.7581 ms
preprocess time = 0.381304; infer time = 2.66086; postprocess time = 0.026384
yolo.infer time: 10.5555 ms
preprocess time = 0.527488; infer time = 2.61504; postprocess time = 0.030552
yolo.infer time: 10.7029 ms
preprocess time = 0.255576; infer time = 2.65344; postprocess time = 0.026216
yolo.infer time: 10.5969 ms
preprocess time = 1.19168; infer time = 2.62579; postprocess time = 0.024984
yolo.infer time: 10.8065 ms
preprocess time = 0.318584; infer time = 2.67699; postprocess time = 0.102008
yolo.infer time: 14.5973 ms
preprocess time = 0.26504; infer time = 3.61933; postprocess time = 0.025968
yolo.infer time: 13.8007 ms
preprocess time = 0.296144; infer time = 3.42707; postprocess time = 0.025992
yolo.infer time: 14.4194 ms
preprocess time = 0.292088; infer time = 3.58272; postprocess time = 0.029744
yolo.infer time: 10.6508 ms
preprocess time = 0.258448; infer time = 2.64038; postprocess time = 0.028544
yolo.infer time: 10.6564 ms
preprocess time = 0.4362; infer time = 2.64269; postprocess time = 0.025152
yolo.infer time: 10.6201 ms
preprocess time = 0.52708; infer time = 2.62349; postprocess time = 0.026736
yolo.infer time: 10.6688 ms
preprocess time = 0.245944; infer time = 2.63962; postprocess time = 0.026792
yolo.infer time: 10.6288 ms
preprocess time = 0.604488; infer time = 2.62298; postprocess time = 0.027056
yolo.infer time: 10.6854 ms
preprocess time = 0.417888; infer time = 2.64038; postprocess time = 0.025704
yolo.infer time: 10.687 ms
preprocess time = 0.24964; infer time = 2.64986; postprocess time = 0.028336
yolo.infer time: 10.6461 ms
preprocess time = 0.417704; infer time = 2.63757; postprocess time = 0.02608
yolo.infer time: 10.5863 ms
preprocess time = 0.51976; infer time = 2.62298; postprocess time = 0.026888
yolo.infer time: 10.7062 ms
preprocess time = 0.244928; infer time = 2.65549; postprocess time = 0.026104
yolo.infer time: 10.9809 ms
preprocess time = 0.724624; infer time = 2.59738; postprocess time = 0.0272
yolo.infer time: 10.6711 ms
preprocess time = 0.254456; infer time = 2.6455; postprocess time = 0.0266
yolo.infer time: 10.7115 ms
preprocess time = 0.246016; infer time = 2.64704; postprocess time = 0.025912
yolo.infer time: 10.5605 ms
preprocess time = 0.642432; infer time = 2.61376; postprocess time = 0.027496
yolo.infer time: 10.5903 ms
preprocess time = 0.70676; infer time = 2.62554; postprocess time = 0.026128
yolo.infer time: 10.655 ms
preprocess time = 0.247392; infer time = 2.63347; postprocess time = 0.028544
yolo.infer time: 10.5942 ms
preprocess time = 0.544168; infer time = 2.62326; postprocess time = 0.02676
yolo.infer time: 10.8423 ms
preprocess time = 0.63004; infer time = 2.60634; postprocess time = 0.028472
yolo.infer time: 10.6226 ms
preprocess time = 0.2466; infer time = 2.6327; postprocess time = 0.025912
yolo.infer time: 10.5913 ms
preprocess time = 0.671624; infer time = 2.62426; postprocess time = 0.027352
yolo.infer time: 10.6326 ms
preprocess time = 0.526512; infer time = 2.62502; postprocess time = 0.026384
yolo.infer time: 10.6927 ms
preprocess time = 0.247336; infer time = 2.65114; postprocess time = 0.0292
yolo.infer time: 10.679 ms
preprocess time = 0.26068; infer time = 2.64653; postprocess time = 0.029232
yolo.infer time: 10.5535 ms
preprocess time = 0.541272; infer time = 2.61325; postprocess time = 0.026936
yolo.infer time: 10.6464 ms
preprocess time = 0.367248; infer time = 2.6391; postprocess time = 0.026648
yolo.infer time: 10.6045 ms
preprocess time = 0.667376; infer time = 2.62707; postprocess time = 0.02964
yolo.infer time: 12.8938 ms
preprocess time = 0.25624; infer time = 3.20154; postprocess time = 0.029128
yolo.infer time: 14.1837 ms
preprocess time = 0.312104; infer time = 3.37075; postprocess time = 0.02936
yolo.infer time: 10.6723 ms
preprocess time = 0.266152; infer time = 2.64627; postprocess time = 0.026304
yolo.infer time: 10.5341 ms
preprocess time = 0.526904; infer time = 2.60838; postprocess time = 0.02608
yolo.infer time: 10.9445 ms
preprocess time = 1.41348; infer time = 2.58918; postprocess time = 0.02792
yolo.infer time: 11.6869 ms
preprocess time = 0.282008; infer time = 2.89562; postprocess time = 0.0248
yolo.infer time: 13.0565 ms
preprocess time = 0.2624; infer time = 3.24224; postprocess time = 0.037504
yolo.infer time: 11.7787 ms
preprocess time = 0.28672; infer time = 2.92275; postprocess time = 0.029424
yolo.infer time: 11.7919 ms
preprocess time = 0.778952; infer time = 2.92582; postprocess time = 0.025592
yolo.infer time: 12.5213 ms
preprocess time = 0.333248; infer time = 3.10912; postprocess time = 0.025112
yolo.infer time: 12.3559 ms
preprocess time = 0.272152; infer time = 3.06688; postprocess time = 0.024448
yolo.infer time: 14.4646 ms
preprocess time = 0.300728; infer time = 3.59452; postprocess time = 0.025096
yolo.infer time: 12.2907 ms
preprocess time = 1.29474; infer time = 2.97446; postprocess time = 0.025528
yolo.infer time: 10.653 ms
preprocess time = 0.474408; infer time = 2.64269; postprocess time = 0.039704
yolo.infer time: 10.6857 ms
preprocess time = 0.255224; infer time = 2.64781; postprocess time = 0.025072
yolo.infer time: 10.6582 ms
preprocess time = 0.255008; infer time = 2.64218; postprocess time = 0.025912
yolo.infer time: 10.9249 ms
preprocess time = 0.743424; infer time = 2.58408; postprocess time = 0.025288
yolo.infer time: 10.6613 ms
preprocess time = 0.36688; infer time = 2.64346; postprocess time = 0.02564
yolo.infer time: 10.6784 ms
preprocess time = 0.25868; infer time = 2.64525; postprocess time = 0.025784
yolo.infer time: 10.6429 ms
preprocess time = 0.38572; infer time = 2.63808; postprocess time = 0.024704
yolo.infer time: 10.9663 ms
preprocess time = 0.690376; infer time = 2.5943; postprocess time = 0.024744
yolo.infer time: 10.6605 ms
preprocess time = 0.272856; infer time = 2.63936; postprocess time = 0.024472
yolo.infer time: 10.9275 ms
preprocess time = 0.729336; infer time = 2.58534; postprocess time = 0.0258
yolo.infer time: 10.9139 ms
preprocess time = 0.74592; infer time = 2.58074; postprocess time = 0.02884
yolo.infer time: 10.6694 ms
preprocess time = 0.25072; infer time = 2.64448; postprocess time = 0.024568
yolo.infer time: 10.5604 ms
preprocess time = 0.537512; infer time = 2.61709; postprocess time = 0.029008
yolo.infer time: 10.8693 ms
preprocess time = 0.685504; infer time = 2.57152; postprocess time = 0.029384
yolo.infer time: 10.6115 ms
preprocess time = 0.372488; infer time = 2.6266; postprocess time = 0.027688
yolo.infer time: 10.7354 ms
preprocess time = 0.637304; infer time = 2.5879; postprocess time = 0.02932
yolo.infer time: 13.2191 ms
preprocess time = 0.34868; infer time = 3.27245; postprocess time = 0.025416
yolo.infer time: 13.1376 ms
preprocess time = 0.285936; infer time = 3.26221; postprocess time = 0.025176
yolo.infer time: 13.8364 ms
preprocess time = 0.297656; infer time = 3.43706; postprocess time = 0.025704
yolo.infer time: 15.2572 ms
preprocess time = 0.28704; infer time = 3.78829; postprocess time = 0.025032
yolo.infer time: 10.5618 ms
preprocess time = 1.14221; infer time = 2.61043; postprocess time = 0.033816
yolo.infer time: 10.6962 ms
preprocess time = 0.288016; infer time = 2.65062; postprocess time = 0.028096
yolo.infer time: 10.6692 ms
preprocess time = 0.255192; infer time = 2.64474; postprocess time = 0.028864
yolo.infer time: 10.606 ms
preprocess time = 0.532144; infer time = 2.62835; postprocess time = 0.02648
yolo.infer time: 10.6775 ms
preprocess time = 0.252592; infer time = 2.64704; postprocess time = 0.043256
yolo.infer time: 10.5046 ms
preprocess time = 0.557888; infer time = 2.60147; postprocess time = 0.028512
yolo.infer time: 10.668 ms
preprocess time = 0.244048; infer time = 2.64397; postprocess time = 0.026936
yolo.infer time: 10.6275 ms
preprocess time = 0.29; infer time = 2.63398; postprocess time = 0.024584
yolo.infer time: 10.6194 ms
preprocess time = 0.263368; infer time = 2.63219; postprocess time = 0.02528
yolo.infer time: 10.7704 ms
preprocess time = 0.263936; infer time = 2.65472; postprocess time = 0.026056
yolo.infer time: 10.6975 ms
preprocess time = 0.261312; infer time = 2.65216; postprocess time = 0.028648
yolo.infer time: 10.6366 ms
preprocess time = 0.256528; infer time = 2.63603; postprocess time = 0.027144
yolo.infer time: 10.8036 ms
preprocess time = 0.712824; infer time = 2.60147; postprocess time = 0.02716
yolo.infer time: 10.5636 ms
preprocess time = 0.693512; infer time = 2.61734; postprocess time = 0.027088
yolo.infer time: 10.8956 ms
preprocess time = 0.68488; infer time = 2.57718; postprocess time = 0.0262
yolo.infer time: 10.9594 ms
preprocess time = 0.679952; infer time = 2.592; postprocess time = 0.029496
yolo.infer time: 10.9157 ms
preprocess time = 0.67844; infer time = 2.58611; postprocess time = 0.025928
yolo.infer time: 10.8969 ms
preprocess time = 0.676832; infer time = 2.57973; postprocess time = 0.02608
yolo.infer time: 10.9578 ms
preprocess time = 1.41248; infer time = 2.59123; postprocess time = 0.038736
yolo.infer time: 10.6464 ms
preprocess time = 0.279432; infer time = 2.63962; postprocess time = 0.029208
yolo.infer time: 10.6757 ms
preprocess time = 0.24676; infer time = 2.64653; postprocess time = 0.02872
yolo.infer time: 10.6881 ms
preprocess time = 0.424576; infer time = 2.64576; postprocess time = 0.036232
yolo.infer time: 14.3484 ms
preprocess time = 0.279768; infer time = 3.56531; postprocess time = 0.026336
yolo.infer time: 10.6728 ms
preprocess time = 0.250088; infer time = 2.64525; postprocess time = 0.025744
yolo.infer time: 10.942 ms
preprocess time = 1.3241; infer time = 2.59021; postprocess time = 0.0298
yolo.infer time: 10.7845 ms
preprocess time = 0.266168; infer time = 2.67366; postprocess time = 0.049472
yolo.infer time: 10.6981 ms
preprocess time = 0.259384; infer time = 2.64295; postprocess time = 0.028232
yolo.infer time: 14.7619 ms
preprocess time = 0.26448; infer time = 3.66822; postprocess time = 0.03604
yolo.infer time: 27.9887 ms
preprocess time = 0.950792; infer time = 6.96755; postprocess time = 0.054464
yolo.infer time: 28.2193 ms
preprocess time = 0.553728; infer time = 7.01901; postprocess time = 0.035896
yolo.infer time: 26.6956 ms
preprocess time = 0.625048; infer time = 6.63782; postprocess time = 0.035536
yolo.infer time: 26.0516 ms
preprocess time = 0.55936; infer time = 6.48602; postprocess time = 0.023296
yolo.infer time: 12.953 ms
preprocess time = 0.292568; infer time = 3.21614; postprocess time = 0.024584
yolo.infer time: 13.0599 ms
preprocess time = 1.08011; infer time = 3.09888; postprocess time = 0.023832
yolo.infer time: 9.6395 ms
preprocess time = 0.838752; infer time = 2.38618; postprocess time = 0.02528
yolo.infer time: 11.0133 ms
preprocess time = 0.549544; infer time = 2.63142; postprocess time = 0.026808
yolo.infer time: 9.8191 ms
preprocess time = 0.392232; infer time = 2.42406; postprocess time = 0.024816
yolo.infer time: 11.4994 ms
preprocess time = 0.334584; infer time = 2.85261; postprocess time = 0.026664
yolo.infer time: 13.3364 ms
preprocess time = 0.268128; infer time = 3.31238; postprocess time = 0.027288
yolo.infer time: 9.7322 ms
preprocess time = 0.49556; infer time = 2.40614; postprocess time = 0.03156
yolo.infer time: 12.9883 ms
preprocess time = 0.307616; infer time = 3.09427; postprocess time = 0.025336
yolo.infer time: 9.9351 ms
preprocess time = 0.640872; infer time = 2.3913; postprocess time = 0.024056
yolo.infer time: 9.7693 ms
preprocess time = 0.44092; infer time = 2.41971; postprocess time = 0.027616
yolo.infer time: 12.554 ms
preprocess time = 0.285744; infer time = 2.97371; postprocess time = 0.027104
yolo.infer time: 12.5067 ms
preprocess time = 0.282712; infer time = 2.98419; postprocess time = 0.0256
yolo.infer time: 10.1005 ms
preprocess time = 0.238472; infer time = 2.5024; postprocess time = 0.029064
yolo.infer time: 10.3225 ms
preprocess time = 0.794056; infer time = 2.44224; postprocess time = 0.026904
yolo.infer time: 10.201 ms
preprocess time = 0.568912; infer time = 2.45862; postprocess time = 0.025952
yolo.infer time: 10.2612 ms
preprocess time = 0.641216; infer time = 2.43277; postprocess time = 0.025792
yolo.infer time: 10.6181 ms
preprocess time = 0.688016; infer time = 2.51162; postprocess time = 0.024344
yolo.infer time: 10.6141 ms
preprocess time = 0.57636; infer time = 2.55232; postprocess time = 0.025744
yolo.infer time: 10.3123 ms
preprocess time = 0.532296; infer time = 2.5559; postprocess time = 0.026424
yolo.infer time: 10.301 ms
preprocess time = 0.728608; infer time = 2.55258; postprocess time = 0.02908
yolo.infer time: 11.2793 ms
preprocess time = 0.81632; infer time = 2.79603; postprocess time = 0.02684
yolo.infer time: 14.3785 ms
preprocess time = 0.311512; infer time = 3.57045; postprocess time = 0.036752
yolo.infer time: 23.6805 ms
preprocess time = 1.45733; infer time = 5.89568; postprocess time = 0.03884
yolo.infer time: 23.6569 ms
preprocess time = 0.881552; infer time = 5.88595; postprocess time = 0.038048
yolo.infer time: 21.4839 ms
preprocess time = 0.673256; infer time = 5.34656; postprocess time = 0.037728
yolo.infer time: 21.4454 ms
preprocess time = 1.77036; infer time = 5.25619; postprocess time = 0.037272
yolo.infer time: 12.2679 ms
preprocess time = 0.275472; infer time = 3.04384; postprocess time = 0.026288
yolo.infer time: 12.9192 ms
preprocess time = 0.272664; infer time = 3.20717; postprocess time = 0.027232
yolo.infer time: 13.3393 ms
preprocess time = 0.31268; infer time = 3.3111; postprocess time = 0.028752
yolo.infer time: 13.4883 ms
preprocess time = 0.281432; infer time = 3.35004; postprocess time = 0.025504
yolo.infer time: 9.9513 ms
preprocess time = 0.71408; infer time = 2.35394; postprocess time = 0.025904
yolo.infer time: 11.4971 ms
preprocess time = 0.520112; infer time = 2.7497; postprocess time = 0.029352
yolo.infer time: 10.1913 ms
preprocess time = 0.342864; infer time = 2.52467; postprocess time = 0.027088
yolo.infer time: 10.4442 ms
preprocess time = 0.79028; infer time = 2.4681; postprocess time = 0.02992
yolo.infer time: 10.1955 ms
preprocess time = 0.225736; infer time = 2.51955; postprocess time = 0.025248
yolo.infer time: 10.1876 ms
preprocess time = 0.227704; infer time = 2.52493; postprocess time = 0.029328
yolo.infer time: 10.0775 ms
preprocess time = 0.504336; infer time = 2.49702; postprocess time = 0.027928
yolo.infer time: 13.8328 ms
preprocess time = 0.655352; infer time = 3.32032; postprocess time = 0.0382
yolo.infer time: 25.501 ms
preprocess time = 0.534312; infer time = 6.35136; postprocess time = 0.042536
yolo.infer time: 22.3024 ms
preprocess time = 0.44784; infer time = 5.55238; postprocess time = 0.039384
yolo.infer time: 24.1893 ms
preprocess time = 0.793928; infer time = 6.02266; postprocess time = 0.03904
yolo.infer time: 23.3918 ms
preprocess time = 0.657456; infer time = 5.82374; postprocess time = 0.038608
yolo.infer time: 13.3024 ms
preprocess time = 0.273632; infer time = 3.30368; postprocess time = 0.027336
yolo.infer time: 9.7877 ms
preprocess time = 0.251176; infer time = 2.42228; postprocess time = 0.025728
yolo.infer time: 9.8527 ms
preprocess time = 0.223856; infer time = 2.43789; postprocess time = 0.026128
yolo.infer time: 10.0498 ms
preprocess time = 0.241552; infer time = 2.4896; postprocess time = 0.118232
yolo.infer time: 9.7782 ms
preprocess time = 0.24208; infer time = 2.41741; postprocess time = 0.0244
yolo.infer time: 10.453 ms
preprocess time = 0.272888; infer time = 2.58995; postprocess time = 0.025
yolo.infer time: 10.4634 ms
preprocess time = 0.298928; infer time = 2.59226; postprocess time = 0.03452
yolo.infer time: 10.4726 ms
preprocess time = 0.2572; infer time = 2.59507; postprocess time = 0.029616
yolo.infer time: 10.4058 ms
preprocess time = 0.256504; infer time = 2.57869; postprocess time = 0.029504
yolo.infer time: 10.4741 ms
preprocess time = 0.299176; infer time = 2.59561; postprocess time = 0.023448
yolo.infer time: 10.3969 ms
preprocess time = 0.247288; infer time = 2.57613; postprocess time = 0.028608
yolo.infer time: 20.6767 ms
preprocess time = 0.377112; infer time = 5.14432; postprocess time = 0.03984
yolo.infer time: 24.2189 ms
preprocess time = 0.785328; infer time = 6.03136; postprocess time = 0.038912
yolo.infer time: 22.3633 ms
preprocess time = 1.97184; infer time = 5.5639; postprocess time = 0.03748
yolo.infer time: 21.4209 ms
preprocess time = 0.8358; infer time = 5.32838; postprocess time = 0.038992
yolo.infer time: 29.5785 ms
preprocess time = 0.649192; infer time = 7.37254; postprocess time = 0.024008
yolo.infer time: 9.8459 ms
preprocess time = 0.282152; infer time = 2.44122; postprocess time = 0.025496
yolo.infer time: 11.1898 ms
preprocess time = 0.297208; infer time = 2.77069; postprocess time = 0.026448
yolo.infer time: 11.5184 ms
preprocess time = 0.264712; infer time = 2.85773; postprocess time = 0.02892
yolo.infer time: 10.7047 ms
preprocess time = 0.274704; infer time = 2.656; postprocess time = 0.024384
yolo.infer time: 10.6322 ms
preprocess time = 0.384464; infer time = 2.63859; postprocess time = 0.024952
yolo.infer time: 9.8145 ms
preprocess time = 0.300048; infer time = 2.43379; postprocess time = 0.03836
yolo.infer time: 9.8616 ms
preprocess time = 0.305432; infer time = 2.44454; postprocess time = 0.023728
yolo.infer time: 9.8949 ms
preprocess time = 0.57316; infer time = 2.38413; postprocess time = 0.028048
yolo.infer time: 10.5507 ms
preprocess time = 0.330976; infer time = 2.61734; postprocess time = 0.02696
yolo.infer time: 13.2993 ms
preprocess time = 0.274768; infer time = 3.3047; postprocess time = 0.028928
yolo.infer time: 11.18 ms
preprocess time = 0.276816; infer time = 2.77428; postprocess time = 0.028752
yolo.infer time: 13.6574 ms
preprocess time = 0.285784; infer time = 3.38662; postprocess time = 0.034632
yolo.infer time: 12.658 ms
preprocess time = 0.28904; infer time = 3.14445; postprocess time = 0.024152

环境:cuda11.1 + tensorrt8.4.1.5+vs2019+opencv3.4.6
编译报错:

严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2019 无法解析的外部符号 "void __cdecl resizeDevice(int const &,float *,int,int,float *,int,int,float,struct utils::AffineMat)" (?resizeDevice@@YAXAEBHPEAMHH1HHMUAffineMat@utils@@@z)，函数 "public: virtual void __cdecl YOLOV8::preprocess(class std::vector<class cv::Mat,class std::allocator > const &)" (?preprocess@YOLOV8@@UEAAXAEBV?$vector@VMat@cv@@v?$allocator@VMat@cv@@@std@@@std@@@z) 中引用了该符号 tensorrt-alpha-yolov8 E:\Vs_Projects_2023\tensorrt-alpha-yolov8\tensorrt-alpha-yolov8\yolov8.obj 1

严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2019 无法解析的外部符号 "void __cdecl hwc2chwDevice(int const &,float *,int,int,float *,int,int)" (?hwc2chwDevice@@YAXAEBHPEAMHH1HH@Z)，函数 "public: virtual void __cdecl YOLOV8::preprocess(class std::vector<class cv::Mat,class std::allocator > const &)" (?preprocess@YOLOV8@@UEAAXAEBV?$vector@VMat@cv@@v?$allocator@VMat@cv@@@std@@@std@@@z) 中引用了该符号 tensorrt-alpha-yolov8 E:\Vs_Projects_2023\tensorrt-alpha-yolov8\tensorrt-alpha-yolov8\yolov8.obj 1

严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2019 无法解析的外部符号 "void __cdecl nmsDeviceV1(struct utils::InitParameter,float *,int,int,int)" (?nmsDeviceV1@@YAXUInitParameter@utils@@PEAMHHH@Z)，函数 "public: virtual void __cdecl YOLOV8::postprocess(class std::vector<class cv::Mat,class std::allocator > const &)" (?postprocess@YOLOV8@@UEAAXAEBV?$vector@VMat@cv@@v?$allocator@VMat@cv@@@std@@@std@@@z) 中引用了该符号 tensorrt-alpha-yolov8 E:\Vs_Projects_2023\tensorrt-alpha-yolov8\tensorrt-alpha-yolov8\yolov8.obj 1

严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2001 无法解析的外部符号 "public: virtual void __cdecl yolo::YOLO::reset(void)" (?reset@YOLO@yolo@@UEAAXXZ) tensorrt-alpha-yolov8 E:\Vs_Projects_2023\tensorrt-alpha-yolov8\tensorrt-alpha-yolov8\yolov8.obj 1

严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2019 无法解析的外部符号 "void __cdecl yolov8::transposeDevice(struct utils::InitParameter,float *,int,int,int,float *,int,int)" (?transposeDevice@yolov8@@YAXUInitParameter@utils@@PEAMHHH1HH@Z)，函数 "public: virtual void __cdecl YOLOV8::postprocess(class std::vector<class cv::Mat,class std::allocator > const &)" (?postprocess@YOLOV8@@UEAAXAEBV?$vector@VMat@cv@@v?$allocator@VMat@cv@@@std@@@std@@@z) 中引用了该符号 tensorrt-alpha-yolov8 E:\Vs_Projects_2023\tensorrt-alpha-yolov8\tensorrt-alpha-yolov8\yolov8.obj 1

报了28个类似的错误

尝试直接在cmd里用cmake编译也失败:
CMake Error at CMakeLists.txt:6 (PROJECT):
No CMAKE_CUDA_COMPILER could be found.

编译过程没问题，使用时报错
启动命令.\trt.exe --model=1.trt --size=640 --batch_size=1 --img=1.jpg --show --savePath=../

[07/21/2023-13:29:26] [I] model_path = 1.trt
[07/21/2023-13:29:26] [I] size = 640
[07/21/2023-13:29:26] [I] batch_size = 1
[07/21/2023-13:29:26] [I] image_path = 1.jpg
[07/21/2023-13:29:26] [I] is_show = 1
[07/21/2023-13:29:26] [I] save_path = ../
[07/21/2023-13:29:27] [I] [TRT] Loaded engine size: 16 MiB
[07/21/2023-13:29:27] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +12, now: CPU 0, GPU 12 (MiB)
[07/21/2023-13:29:27] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +148, now: CPU 0, GPU 160 (MiB)
[07/21/2023-13:29:27] [W] [TRT] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
[07/21/2023-13:29:27] [E] [TRT] 3: [runtime.cpp::nvinfer1::Runtime::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime.cpp::nvinfer1::Runtime::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[07/21/2023-13:29:27] [I] the engine's info:
[07/21/2023-13:29:27] [I] idx = 0, images: -1, 3, 640, 640,
[07/21/2023-13:29:27] [I] idx = 1, output0: -1, 6, 8400,
[07/21/2023-13:29:27] [I] the context's info:
[07/21/2023-13:29:27] [I] idx = 0, images: 1, 3, 640, 640,
[07/21/2023-13:29:27] [I] idx = 1, output0: 1, 6, 8400,
[07/21/2023-13:29:27] [I] 1

后面是否会支持yolov8的segment实例分割？

void YOLOV8::preprocess(const std::vector<cv::Mat>& imgsBatch)
{
    // 1.resize
    resizeDevice(m_param.batch_size, m_input_src_device, m_param.src_w, m_param.src_h,
        m_input_resize_device, m_param.dst_w, m_param.dst_h, 114, m_dst2src);

Also, I don't quite understand the principle of how to construct utils: AffineMat matrix. Isn't the image affine transformation a 3x3 matrix with 6 unknowns. Is it possible to use a 2x2 scaling matrix. I don't quite understand the meaning of the third column of numbers, or rather, I don't quite understand this matrix.

I am having trouble running the NMS function in the TensorRT-Alpha repository. I am getting an error message related to CUDA support, and I am not sure what the issue is. Can you please provide guidance on how to resolve this issue? Thank you.

遇到这样一种情况：yolov6模型转换后推理，但是没有检测框出来，全部检测结果为无目标。
另外试了下yolov6n.onnx转换到trt，发现也是一样的没有目标框。
本来是以为是tensorrt和cuda/cudnn的版本有冲突引起的，但是发现没有冲突的情况下，也是一样的现象。

I want to change three channel RGB image into single channel Gray image.
Can you help me? thank you!

报错信息：
terminate called after throwing an instance of 'thrust::system::system_error'
what(): radix_sort: failed on 2nd step: cudaErrorHostMemoryNotRegistered: pointer does not correspond to a registered memory region
Aborted (core dumped)

解决方案：
utils/yolo.cpp文件中，原始代码：

utils/yolo.cpp文件中，更改为：

飞哥已经同步更新代码，再次感谢飞哥：

Hello, the first time I use your project, I use YOLOv5, but in the process of compiling with VS, there are a lot of errors, I wonder if some configuration files are missing? Below are the files included in the project and the error images

不使用trtexec ，能否把onnx文件 转为trt文件，进行tensorRT C++ 推理呢？有些情况是，不能使用trtexec的，通过pip 安装的tensorRT 环境没有trtexec~

/usr/bin/ld: libyolov8.so: undefined reference to `sample::splitToStringVec(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, char)'

有人报这个错误吗

Thanks for this great repository. We have trained our custom dataset with Yolov7 and converted to onnx/trt with the following command:
python export.py --weights path to trained.pt --dynamic --grid
trtexec.exe --onnx=name.onnx --saveEngine=name.trt --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 --buildOnly --fp16
We are using TensorRT-8.4.2.4, and cu113. It runs great most time, but sometime, num_boxes = -2147483648 and got read access violation error at int keep_flag = ptr[6] in postprocess method as shown below.
`
int num_boxes = std::min((int)(m_output_objects_host + bi * (m_param.topK * m_output_objects_width + 1))[0], m_param.topK);

    for (size_t i = 0; i < num_boxes; i++)
    {
        float* ptr = m_output_objects_host + bi * (m_param.topK * m_output_objects_width + 1) + m_output_objects_width * i + 1;
        int keep_flag = ptr[6];
        if (keep_flag)
        {
            float x_lt = m_dst2src.v0 * ptr[0] + m_dst2src.v1 * ptr[1] + m_dst2src.v2; // left & top
            float y_lt = m_dst2src.v3 * ptr[0] + m_dst2src.v4 * ptr[1] + m_dst2src.v5;
            float x_rb = m_dst2src.v0 * ptr[2] + m_dst2src.v1 * ptr[3] + m_dst2src.v2; // right & bottom
            float y_rb = m_dst2src.v3 * ptr[2] + m_dst2src.v4 * ptr[3] + m_dst2src.v5;
            m_objectss[bi].emplace_back(x_lt, y_lt, x_rb, y_rb, ptr[4], (int)ptr[5]); //
        }
    }

`
What could cause this exception?

当采用批次为9推理时，只有第一张有结果，其余都是没有结果，获得坐标为0的

yolov7 的代码中运行过程中出现Error Code 1: Cuda Runtime (an illegal memory access was encountered)，
想咨询一下作者您用所用pt 文件是哪个然后怎么转成trt文件？

环境说明

使用笔记本的3060显卡, cuda11.2, cudnn8.1.1
项目的trt文件是从基于yolov8n-seg的模型训练得到的pt文件
经过YOLOv8-TensorRT项目的export-seg.py转换得到onnx: python .\export-seg.py --weights best.pt --opset 11 --sim --input-shape 1 3 1280 1280 --device cuda:0

然后再通过 trtexec 得到的: trtexec.exe --onnx=best.onnx --saveEngine=yolov8n-seg.trt
模型的输入 img_size 为 1280x1280

错误日志

./yolov8.exe  --model="D:\file_sum\python\highway_defect_yolov8\model_data\yolov8n-seg.trt" --size=1280 --batch_size=1  --img="D:\file_sum\dataset\all\test.jpg" --show
[03/09/2023-19:07:12] [I] model_path = D:\file_sum\python\highway_defect_yolov8\model_data\yolov8n-seg.trt
[03/09/2023-19:07:12] [I] size = 1280
[03/09/2023-19:07:12] [I] batch_size = 1
[03/09/2023-19:07:12] [I] image_path = D:\file_sum\dataset\all\test.jpg
[03/09/2023-19:07:12] [I] is_show = 1
[03/09/2023-19:07:13] [I] [TRT] [MemUsageChange] Init CUDA: CPU +355, GPU +0, now: CPU 9712, GPU 1247 (MiB)
[03/09/2023-19:07:13] [I] [TRT] Loaded engine size: 377 MiB
[03/09/2023-19:07:14] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +747, GPU +264, now: CPU 10529, GPU 1889 (MiB)
[03/09/2023-19:07:14] [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.1.1
[03/09/2023-19:07:14] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +379, now: CPU 0, GPU 379 (MiB)
[03/09/2023-19:07:14] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 10528, GPU 1894 (MiB)
[03/09/2023-19:07:14] [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.1.1
[03/09/2023-19:07:14] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +481, now: CPU 0, GPU 860 (MiB)
[03/09/2023-19:07:14] [I] the engine's info:
[03/09/2023-19:07:14] [I] idx = 0, images: 1, 3, 1280, 1280,
[03/09/2023-19:07:14] [I] idx = 1, outputs: 1, 33600, 38,
[03/09/2023-19:07:14] [I] the context's info:
[03/09/2023-19:07:14] [I] idx = 0, images: 1, 3, 1280, 1280,
[03/09/2023-19:07:14] [I] idx = 1, outputs: 1, 33600, 38,
[03/09/2023-19:07:14] [I] 1
[03/09/2023-19:07:16] [E] [TRT] 1: [executionContext.cpp::nvinfer1::rt::ExecutionContext::executeInternal::667] Error Code 1: Cuda Runtime (an illegal memory access was encountered)

尝试解决

尝试过的方案:
①cmake中架构添加了一个86, set_property(TARGET ${PROJECT_NAME} PROPERTY CUDA_ARCHITECTURES 60 61 62 70 72 75 86) 然后重新编译工程, 但是仍然没用...
②减小推理时的图像大小--size=640... 后来想想也不对, python是可以进行正确推理的...

您好，release x64下能编译出来exe，运行没有问题；但是debug下存在图上报错，不知道为啥？

hi,@ @FeiYull
i compile your yolov8 demo, on windows with jetbrains CLion,I got an error:
D:\TensorRT-8.2.5.1\samples\common\sampleUtils.h(170): error: more than one operator "<<" matches these operands:
function "std::basic_ostream<_Elem, _Traits>::operator<<(__nv_bool) [with _Elem=char, _Traits=std::char_traits]"

        operand types are: std::basic_ostream<char, std::char_traits<char>> << const __half
      detected during instantiation of "void sample::dumpBuffer<T>(const void *, const std::string &, std::ostream &, const nvinfer1::Dims &, const nvinfer1::Dims &, int32_t, int32_t) [with T=__half]" 

(257): here

1 error detected in the compilation of "D:/Object_Detection/tensorrt-alpha/yolov8/decode_yolov8.cu".
decode_yolov8.cu

my tensorrt version is 8.2.5.1,
how can i resolve it? please help me!

I am unable to export the ONNX file and the process seems to be stuck on the conversion step. Please provide assistance in resolving this issue.

I am trying to use the YOLOv4 model provided in the TensorRT-Alpha repository, but I am encountering an error when trying to run the inference. Specifically, I am getting the following message: 'CUDA error: invalid device function'. I have double-checked that my CUDA version is compatible with the code and that my GPU is properly configured. I would appreciate any help in resolving this issue.

当我按如下操作执行时：
git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha/yolov8
mkdir build
cd build
cmake ..

cmake 报错：
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
TRT_NVCAFFE_PARSER
linked by target "yolov8" in directory E:/software/anconda/.conda/envs/yolov8/tensorrt-alpha/yolov8
linked by target "app_yolov8" in directory E:/software/anconda/.conda/envs/yolov8/tensorrt-alpha/yolov8

cmake版本已更新至最新版，此error仍存在，不知该如何解决？

How to implement yolov8's tensorRT int8 quantization?