Memory leak in PoseNet's video processing pattern about ml5-next-gen HOT 5 OPEN

Hi @davepagurek thank you for reporting this and for your patience! I am working adding some documentation to this repo and archiving it. We have moved current ml5.js development to a new repository:

https://github.com/ml5js/ml5-next-gen

The latest examples are here:

https://editor.p5js.org/ml5/collections/pUzWMkdmE

We are planning to launch a new version of the library + website in June.

The new BodyPose functionality of ml5.js uses updated models behind the scenes (MoveNet and BlazePose). @ziyuan-linn have you noticed any signs of a memory leak like this in our newer examples?

from ml5-next-gen.

Just skimming the code, it looks like the looping pattern here https://github.com/ml5js/ml5-next-gen/blob/main/src%2FBodyPose%2Findex.js#L312 would avoid the issue since it doesn't return the result of the promise, but I can try profiling the examples when I get back to my computer to be sure.

from ml5-next-gen.

I was testing out this one https://editor.p5js.org/ml5/sketches/OukJYAJAb and taking heap snapshots every few minutes, and the snapshots seem to increase in size over time:

So that might mean there's a leak somewhere still? Here's the first and last snapshot for comparison:
BEFORE-Heap-20240504T094341.heapsnapshot.zip
AFTER-Heap-20240504T094350.heapsnapshot.zip

I see something kind of similar going on in the added Objects. They don't appear as promises this time, but the last one in the chain does say "pending."

from ml5-next-gen.

Thanks @davepagurek I moved this issue into the current repo for further discussion and tracking!

from ml5-next-gen.

@davepagurek Thank you for reporting this issue!

I did some testing and found memory leaks in a few models. Here are the heap snapshots:

bodyPose, with MoveNet as the underlying model:

bodyPose, with BlazePose as the underlying model on mediapipe runtime:

faceMesh, on tfjs runtime:

faceMesh, on mediapipe runtime:

This problem only seems to occur when using the mediapipe runtime. I think the problem could be due to MediaPipe using WASM and C++(no automatic garbage collection). Even this tfjs demo of face-landmark-detection on MediaPipe runtime have a memory leak. I could not find much information about this, but here is a potentially related issue with a solution.

For now, switching to tfjs runtime or MoveNet on the new version of ml5 should fix the memory leak:

bodyPose = ml5.bodyPose("BlazePose", { runtime: "tfjs" });

or

bodyPose = ml5.bodyPose("MoveNet");

from ml5-next-gen.