How can I reduce GPU memory usage in generation? about flowmdm HOT 2 CLOSED

Hi @vegelate,

Thank you for your interest in our work! Can you give me more details on what you are trying to run? I was able to perform the inference for the composition example on Babel in a RTX 3050 with 4Gb of RAM.

That said, the denoising with the HumanML3D example is highly memory-hungry because the length of the motion is ~4-5 minutes. For such long sequences, the memory bottleneck is in the APE stage (as long as you use the --use_chunked_att flag). I will implement next week a more memory-efficient version. Right now, during APE stage, the whole sequence (N frames) is denoised all at once, and the attention memory complexity is N². This can be improved by denoising all subsequences in parallel since the attention is limited anyway to each intra-subsequence segment. In that case, the memory complexity will be M² where M is the maximum length of all subsequences (M << N). That should decrease the memory requirements for very long compositions.

Stay tuned!

from flowmdm.

Thank you for your reply, I tried fewer animation sequences, and it works now.
The quality of the animation generated is very high, outstanding work!

from flowmdm.