RetinaFace not utilizing more than 1GB of GPU memory!! Any ideas on how to utilize more of GPU?? about tensorrtx HOT 5 CLOSED

Here 1 << 20 means 1MB, which is the workspace size for tensorrt engine building phase.
This one will not impact the runtime GPU memory utilization.

from tensorrtx.

Okay! So, I am using RetinaFace for inference on a video. And it utilizes only 1000 MB during inference. As a result of which processes only at 25 fps. Which is very low for the standards of TensorRT. Can we somehow increase the runtime GPU memory utilization or the 20 fps processing speed??

from tensorrtx.

You can try smaller input size to get faster speed.

from tensorrtx.

Hey @wang-xinyu

Is there a way we can allot more GPU memory to the engine or force the engine to use higher memory?

from tensorrtx.

@aditya-dl I didn't see any TensorRT API can do this.
I think tensorrt is merging the layers and channels during building phase, then calling cudnn and cuda functions to do inference during runtime phase. It aims at saving the GPU memory, and get shorter latency at the same time.
If you want to use more GPU memory to get shorter latency, I think it can't.
If you want to use more GPU memory to get more FPS, you can try increasing the batch size to processing multi images a time.

from tensorrtx.