Comments (12)
pip install onnxruntime-gpu
from comfyui-liveportraitkj.
For me it uses 100% cpu + 100% gpu and around 2.5GB v-ram the entier generation time with a 832x1152 resolution image. but it generates pretty quickly, 3060ti.
And there's even more to it than that. Video with longer duration sucked a lot of CPU and RAM resources. My RAM was eating up almost all 29 gigabytes and the computer was freezing. I used "pip install onnxruntime-gpu" and it became not much easier, but with long videos, the RAM clogged and I'm afraid there may be everything. I have a 360rtx/12 video card, 32 RAM.
from comfyui-liveportraitkj.
I agree heavily loads the processor, as well as after RAM! I would like to note here is another thing, if you load not such a large image and do not choose a fully long video will not be much easier, perhaps reduce upscale
from comfyui-liveportraitkj.
Already install onnxruntime-gpu, but it seems sometime it consume cpu,sometime it cost GPU. I will keep on watching it to find out the reason.
from comfyui-liveportraitkj.
Overall I wonder about the performance. The paper claims to achieve 12.8 ms per frame, but in my case it's far from it and for 3/4 of the time it's not even utilizing the GPU, nor a CPU (both are at 20 and 10% respectively), so wonder apart from the onnx issue, is there anything else that could be a bottleneck - looks like a single core process is running and blocking the whole thing.
from comfyui-liveportraitkj.
I don't have the issue so I have no input beyond onnx, but after getting that going my 4090 is at 55% steadily during the run. For reference. CPU on process is %345. Without GPU it was much higher CPU.
from comfyui-liveportraitkj.
For me it uses 100% cpu + 100% gpu and around 2.5GB v-ram the entier generation time with a 832x1152 resolution image. but it generates pretty quickly, 3060ti.
from comfyui-liveportraitkj.
For me it uses 100% cpu + 100% gpu and around 2.5GB v-ram the entier generation time with a 832x1152 resolution image. but it generates pretty quickly, 3060ti.
And there's even more to it than that. Video with longer duration sucked a lot of CPU and RAM resources. My RAM was eating up almost all 29 gigabytes and the computer was freezing. I used "pip install onnxruntime-gpu" and it became not much easier, but with long videos, the RAM clogged and I'm afraid there may be everything. I have a 360rtx/12 video card, 32 RAM.
Yep, 35seconds example video or i even tried 1minuted can eat all resources that you have and if you don't have enough your pc will freeze for minutes :)) mine was frozen for 15 minutes for a 1min video, again generation is fine, but at the end every single frame of lets say 1minutes video at 24fps have to be proccessed, i mean that created more than a thousand images and eats all your ram. 20second or less is fine, for longer videos we have to limited frame cap and generate couple of videos and join them together later.
from comfyui-liveportraitkj.
Same problem here, I've been trying to run the example video on an A40 instance on Runpod and everything freezes.
from comfyui-liveportraitkj.
On my side, I think it's not really a matter of some bottleneck, I have 128GB RAM, so some frame off-loading is not the problem. Same with vram - 24GB. I do have onnx-gpu installed, but it's 1.5 I believe, maybe there's a version mismatch, but even that wouldn't explain the low load on both CPU and GPU in the pre-processing phase.
Anyway this could work way more efficiently, and given the low utilization numbers provided by others, I think with proper use of both CPU and GPU the claimed 12.8ms per frame is possible, regardless of the length of the video. It could be that this is an issue with comfy itself, that it needs to finish one 'block' from the pre-process node, before moving to the generation.
from comfyui-liveportraitkj.
On my side, I think it's not really a matter of some bottleneck, I have 128GB RAM, so some frame off-loading is not the problem. Same with vram - 24GB. I do have onnx-gpu installed, but it's 1.5 I believe, maybe there's a version mismatch, but even that wouldn't explain the low load on both CPU and GPU in the pre-processing phase.
Anyway this could work way more efficiently, and given the low utilization numbers provided by others, I think with proper use of both CPU and GPU the claimed 12.8ms per frame is possible, regardless of the length of the video. It could be that this is an issue with comfy itself, that it needs to finish one 'block' from the pre-process node, before moving to the generation.
Their code has a lot of inefficiencies, I don't know if the their speed claim is about the whole process or part of it. For example skipping the pasteback gives ~30% speed boost.
For reference the numbers I'm currently getting for video editing in the develop branch with 4090, for the detection/cropping part, using CUDA for onnx: 33it/s
And the rest, which uses mostly GPU but there's also lots of CV2/numpy operations that are done on CPU, I'm getting 12it/s on Ryzen 7950x
So something like ~14 fps without pasteback and ~11 fps with.
from comfyui-liveportraitkj.
Oh, and about the memory issue...that's common in Comfy when the frame count gets really high, it's not really designed to handle that in general as everything is kept in memory with no disk caching.
from comfyui-liveportraitkj.
Related Issues (20)
- liveportrait官方更新了动物模型
- 怎么使用那个动物模型呢? HOT 7
- Very slow compared to other version
- Why did I update the node and even download multiple times to install the latest version, but still fail to select the animal model option when loading the model node. This question has been bothering me for a long time. Has anyone encountered this situation? HOT 3
- ValueError: The number of driving images should be larger than the number of source images. HOT 2
- Updated today and I am getting a "Error when executing VHS_LoadVideo" HOT 4
- Too many frames are not being processed
- VHS Audio from driving video source HOT 5
- Face becomes distorted after recent ComfyUI updates
- Once expression_friendly is set to true, the system reports an error that expression_friendly works only with single source image HOT 1
- Where can I find a way to do higher than 512px generations? HOT 2
- Animal mode HOT 6
- Videos are very shaky and jittery even with stable shoulders and limited movement HOT 1
- Video Length HOT 2
- 抱歉,打扰,您这个还需要额外再安装 comfyUI吗? HOT 3
- raise ValueError("No face detected in FIRST source image") HOT 3
- no audio at the end liveportrait_video_example_02.json HOT 2
- Does LivePortraitKJ support animal lip syncing as well?
- How to tweak the mask settings to reduce flicker/jitter around the mask edges?
- Will this project update to support the new function that img2img to generate photo?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui-liveportraitkj.