Extremely slow. Is facenet's Inception Resnet v1 supported with TensorRT? about facenet_trt HOT 6 CLOSED

Thanks for trying this on Nano. I didn't have a Nano, so not able to do test for you. But I did test on Xavier and its result is same as expected.

First, Inception Resnet v1 and MTCNN both have no full layer support, but TRT could convert non-support layer to TensorFlow. You can check the logs https://github.com/JerryJiaGit/facenet_trt/blob/master/log_xavier_trt5.txt

Second, the init needs few minutes to convert layers, so you could check your log to see if it hang at init or extremely slow at face recognization.

Third, suggest that you could try on one modern NVIDIA dGPU to make sure code is good at first. Nano Tegra has a Maxwell architecture GPU, please check below links for some suggestions:

• https://devtalk.nvidia.com/default/topic/1057812/jetson-nano/optimize-tf-trt-models-on-jetson-nano-to-improve-inference-timing-and-efficiency/
• https://devtalk.nvidia.com/default/topic/1051546/jetson-nano/optimizing-tf-trt-load-time/
• https://jkjung-avt.github.io/tf-trt-on-nano/

from facenet_trt.

Thanks for trying this on Nano. I didn't have a Nano, so not able to do test for you. But I did test on Xavier and its result is same as expected.

First, Inception Resnet v1 and MTCNN both have no full layer support, but TRT could convert non-support layer to TensorFlow. You can check the logs https://github.com/JerryJiaGit/facenet_trt/blob/master/log_xavier_trt5.txt

Second, the init needs few minutes to convert layers, so you could check your log to see if it hang at init or extremely slow at face recognization.

Third, suggest that you could try on one modern NVIDIA dGPU to make sure code is good at first. Nano Tegra has a Maxwell architecture GPU, please check below links for some suggestions:

• https://devtalk.nvidia.com/default/topic/1057812/jetson-nano/optimize-tf-trt-models-on-jetson-nano-to-improve-inference-timing-and-efficiency/
• https://devtalk.nvidia.com/default/topic/1051546/jetson-nano/optimizing-tf-trt-load-time/
• https://jkjung-avt.github.io/tf-trt-on-nano/

Thank you for the quick reply, I really appreciate it :) , I'll check it up and update here when there's any improvements.

from facenet_trt.

Hello @JerryJiaGit

I currently don't have time to test the above suggestions, but I have a small question, hope you'll help me out. Is it right that these methods are TF-TRT method, which is not a pure TensorRT method? As I have known so far, pure TensorRT method includes converting a Tensorflow frozen trained model to a file in .uff format then we use it to create an TensorRT engine which is then used by direct import tensorrt (not import tensorflow.contrib.tensorrt) API to code. And the pure TensorRT method is assumed to be faster than TF-TRT, so why everyone as well as you still uses TF-TRT method? Why don't you use pure TensorRT? Does it have additional advantages on doing that?

from facenet_trt.

To minimize code changes, but get perf improvement, it is the purpose.

from facenet_trt.

To minimize code changes, but get perf improvement, it is the purpose.

Hi @JerryJiaGit , thank you for sharing! I use your code to run on NV Xavier successfully. However, I run ./contributed/real_time_face_recognition.py and got only 4 FPS. Do you know TensorRT engine method to improve speed?
input video: 1x 720p@30fps
SVM classifier trained by 7 persons, total 11 images. Use python3 ./src/classifier.py TRAIN.

from facenet_trt.

To minimize code changes, but get perf improvement, it is the purpose.

Hi @JerryJiaGit , thank you for sharing! I use your code to run on NV Xavier successfully. However, I run ./contributed/real_time_face_recognition.py and got only 4 FPS. Do you know TensorRT engine method to improve speed?
input video: 1x 720p@30fps
SVM classifier trained by 7 persons, total 11 images. Use python3 ./src/classifier.py TRAIN.

4 FPS is too low, did you tried different L4T image for your Xavier? I encountered such low-performance issue with a very old image.

from facenet_trt.