Detection duplicates with fp16 on Jetson Nano (TensorRT v8.2.1.8) about tensorrt-for-yolo-series HOT 2 OPEN

Actually I observed something peculiar. I tried these two different combinations

(yolov7)$ python3.10 export.py --weights my_models/yolov7-tiny.pt --grid --simplify --topk-all 200 --iou-thres 0.1 --conf-thres 0.4 --img-size 416 416
(trt)$ python3.6 export.py -o my_models/yolov7-tiny-416.onnx -e my_models/yolov7-tiny-416-fp16.trt -w 2 --iou_thres 0.1 --conf_thres 0.4 --end2end -p fp16 --max_det 200

and

(yolov7)$ python3.10 export.py --weights my_models/yolov7-tiny.pt --grid --simplify --topk-all 200 --iou-thres 0.7 --conf-thres 0.4 --img-size 416 416
(trt)$ python3.6 export.py -o my_models/yolov7-tiny-416.onnx -e my_models/yolov7-tiny-416-fp16.trt -w 2 --iou_thres 0.7 --conf_thres 0.4 --end2end -p fp16 --max_det 200

expecting that the first combination with the small iou_thres would result in a more permissive model that would allow for multiple detections of the same object, while the second combination would be more conservative and only permit the most dominant detection to survive. To my surprise, the two approached had absolutely no difference, as if the iou_thres flag does not impact the conversion at all.

Any idea why is this happening? Has anyone experienced something similar before?

from tensorrt-for-yolo-series.

Ok, last update. I tried to skip --end2end flag in both conversions (pt to onnx, and onnx to trt) and I set the flag to False like this when I call the inference

classIds, confidences , bboxs   = self.inference(img,ratio,end2end=False)

Then everything works smoothly since the post-processing takes care of the NMS, but the inference time increases; not significantly but it does especially in a platform like the Nano. It's still faster than its onnx counterpart, but I think this approach is sub-optimal. Is there something special with fp16 that forces me to drive this way?

Thanks in advance for your support and I hope this issue will help someone in the future. Cheers

Looking forward to your reply!

from tensorrt-for-yolo-series.