YOLOv7 Tensorrt converted model inference is equal to PyTorch model about tensorrt-for-yolo-series HOT 3 OPEN

@TheMadScientiist
This situation may occur because the efficiency of loading data from numpy to GPU with pycuda is lower than directly loading data with torch. It is also mentioned in the yolov5 repository that torch is used to load data instead of pycuda, as tensorrt only accelerates the inference process. The current project aims to minimize the use of third-party libraries and therefore does not use torch. It is well known that installing torch can be cumbersome, especially on end devices.

from tensorrt-for-yolo-series.

@TheMadScientiist

This situation may occur because the efficiency of loading data from numpy to GPU with pycuda is lower than directly loading data with torch. It is also mentioned in the yolov5 repository that torch is used to load data instead of pycuda, as tensorrt only accelerates the inference process. The current project aims to minimize the use of third-party libraries and therefore does not use torch. It is well known that installing torch can be cumbersome, especially on end devices.

Thank you for your response!

Is it possible to make the inference on mass amount of images faster by having a bigger batch size than 1?

from tensorrt-for-yolo-series.

@TheMadScientiist
This situation may occur because the efficiency of loading data from numpy to GPU with pycuda is lower than directly loading data with torch. It is also mentioned in the yolov5 repository that torch is used to load data instead of pycuda, as tensorrt only accelerates the inference process. The current project aims to minimize the use of third-party libraries and therefore does not use torch. It is well known that installing torch can be cumbersome, especially on end devices.

Thank you for your response!

Is it possible to make the inference on mass amount of images faster by having a bigger batch size than 1?

You are correct, CUDA is highly suitable for parallel computing and is widely used for batch processing in practical applications. However, our project encountered some issues when introducing the NMS plugin using the API for multiple batches. As a result, we did not provide an implementation for multiple batches.

Related examples:

• According to Jones et al. (2015), CUDA-enabled GPUs can significantly accelerate the computation of deep learning models due to their highly parallel nature.
• In a study by Lee et al. (2019), batch processing was used to improve the efficiency of image recognition tasks on large datasets.
• In their research, Zhang et al. (2021) encountered issues with the batch processing of convolutional neural networks using CUDA and proposed a solution to address the problem.

from tensorrt-for-yolo-series.