Comments (13)
Just found a more simple solution for "cuda out of memory" problem while running SimSwap on 2GB GPU:
I only insert a with torch.no_grad(): command in ../util/videoswap.py between lines 48 and 49
(and add 4 more spaces indent to every following line from 49 to 84)
and it works perfect
from simswap.
There are multiple things you could do.
-
Lower the size of your input videos.
-
Split the chunks into separate files, then loop over them or do it one by one (painful).
-
Modify videoswap.py using the below as a starting point.
Line 38 in fc4b701
-
Use a subprocess and use ffmpeg to split the video into chunks, then do a for loop over each video chunk using the python script, then merge them after the fact in a video editor or with ffmpeg. For example:
~pseudo code~
for video_file in video_file_directory:
python test_video_swapsingle.py video_file ...
I would go with number 3, with pseudo code being something like.
this isn't tested, it's just to give you an idea
current_frame = 0
max_frame = 14
for frame_index in tqdm(range(frame_count)):
ret, frame = video.read()
if ret:
current_frame += 1
if current_frame == max_frame:
Do something to empty video memory here
current_frame = 0
detect_results = detect_model.get(frame,crop_size)
if detect_results is not None:
.....
Like I said, I haven't tested it, but it could be a bit of work to get it implemented from scratch as I haven't looked into how the models are loaded into memory yet. The above is definitely enough to work out your own solution though without messing with torch though.
from simswap.
The way torch.cuda.empty_cache() works is it only frees the memory that it's able to. Remember what I said about me not being aware of how the models are loaded into memory with this project? This is what I was referring to. They may be instantiated in different parts of the script, so it may be a bit more work, but you can try what I said below.
Also, you're running that torch call every frame which isn't necessary / can lead to some issues. Also, you don't need to use user input to go to the next iteration. It probably runs out of memory because it's still executing in the background while waiting for your input. Try this instead (untested as I'm away from my machine).
# Add these two lines above the for loop.
current_frame = 0
max_frame = 14
for frame_index in tqdm(range(frame_count)):
ret, frame = video.read()
if ret:
# If ret returns true, increment the current_frame counter by 1.
current_frame += 1
# if the current frame count equals the max frame count, do something.
if current_frame == max_frame:
# Let's empty the cache.
torch.cuda.empty_cache()
# Reset the counter back to 0.
current_frame = 0
from simswap.
Ok.
Added Line #50 in util/videoswap.py
torch.cuda.empty_cache()
This lets me process 99 frames before out of memory....
I'll try to free memory also in the second for loop
from simswap.
Ok.
Added Line #50 in util/videoswap.py
torch.cuda.empty_cache()
This lets me process 99 frames before out of memory....
I'll try to free memory also in the second for loop
Great to hear. I would try to create a little wrapper function where you can tune your own parameters (max frame count), and plug it into line 50 where it executes torch.cuda.empty_cache()
every nth frame.
from simswap.
Yes.
But why does it run out of memory after 99 frames even if i call "empty_cache" after each frame?
I cannot find anything filling the cache. Searched all other scripts in simswap.
Btw.:
I don't know much about python... just beginner after 30 years coding in (visual)basic and a little c++
from simswap.
So I need little bit of help.
I've inserted the following code:
for frame_index in tqdm(range(frame_count)):
torch.cuda.empty_cache()
ret, frame = video.read()
if frame_index == 98:
print (frame_index)
input("Press Enter to continue...")
break
Then it begins to write the video_file(1) containing the first 98 frames
Is there a way to jump back to video_swap but continue with frame 99 for the next 98 frames?
Call video_swap or something like goto video_swap?
After the break it just had to write video_file(2)
And so on...
Don't know if this would work and how to....
EDIT.
Got it to work as written above, but when calling swap_video again (break after 10 frames) it runs out of memory immediately.
from simswap.
The user input is just for some testing purpose after 98 frames.
Resetting frame index to 0 would process the same part of the input video and overwrite the temporary image sequence...
I think I found a solution how to process the whole input via batch and some additional parameters in video_swapsingle.py (start frame, end frame) without the need to split it into shorter parts.
But I have a daytime job now.....
from simswap.
Resetting frame index to 0 would process the same part of the input video and overwrite the temporary image sequence...
Read my proposed code again please. It's not about setting the frame index to 0, it's about creating a separate counter variable that increases a certain amount of times in the loop, and once it hits a certain limit (max_frame), the counter resets.
You said that your GPU runs out of memory every 15th frame or so. In theory, clearing your GPU cache via torch's methods (which may not work) or doing something to alleviate GPU resources every 15 frames would prevent you from doing all of the extra steps you've mentioned.
from simswap.
Looks like i accidently got it to work on 2GB GPU VRAM....
Not the way I initially planned .... but it works.
No problem processing testvideo duration 23 sec. / 1396 frames.
As soon as I have cleaned up the code I will post the changes I've made.
(test_video_swapsingle.py / videoswap.py / test_options.py)
EDIT:
I will write a simple GUI (VB6 :-)
from simswap.
Here are the changes I made to run SimSwap on 2GB VRAM:
./options/test_options.py
self.parser.add_argument("--first_frame", dest="first_frame", type=int, default=0, help="Set frame to start from.")
./util/videoswap.py
.
.
.
from util.add_watermark import watermark_image
#frame_index = 0
first_frame = 0
.
.
.
def video_swap(first_frame , video_path, id_vetor, swap_model, detect_model, save_path, temp_results_dir='./temp_results', crop_size=224, no_simswaplogo = False):
.
.
.
for frame_index in tqdm(range(first_frame,frame_count)):
torch.cuda.empty_cache()
ret, frame = video.read()
if frame_index == 1:
break
.
.
.
video.release()
if frame_index > 1:
image_filename_list = []
path = os.path.join(temp_results_dir,'*.jpg')
image_filenames = sorted(glob.glob(path))
clips = ImageSequenceClip(image_filenames,fps = fps)
test_video_swapsingle.py
.
.
.
first_frame = 0
video_swap(first_frame, opt.video_path, latend_id, model, app,
opt.output_path,temp_results_dir=opt.temp_path,no_simswaplogo=opt.no_simswaplogo)
first_frame = 2
video_swap(first_frame, opt.video_path, latend_id, model, app, opt.output_path,temp_results_dir=opt.temp_path,no_simswaplogo=opt.no_simswaplogo)
test_video_swapsingle.py calls video_swap in ./util/videoswap.py and processes the first 2 frames before the break
then it calls video_swap again, starting at frame 2 and runs until the end of the input file. Tested so far on 2600 frames but there seems to be no limit.
torch.cuda.empty_cache() clears the VRAM before processing every single frame...
Not perfect but (working for me).....
from simswap.
Just found a more simple solution for "cuda out of memory" problem while running SimSwap on 2GB GPU:
I only insert a with torch.no_grad(): command in ../util/videoswap.py between lines 48 and 49
(and add 4 more spaces indent to every following line from 49 to 84)and it works perfect
OMG, I forgot to add this .I will make it done in the next update.
from simswap.
:-)
Came across it making some more mods to first order motion model and co-part segmentation
from simswap.
Related Issues (20)
- Consider opening an issue on the model's GitHub repository to see if it can be updated to use a recent version of Cog
- New Issue about:Pretrained network G has fewer layers HOT 2
- One person training. HOT 1
- Why didn't you change your face? It's just a mask / 为什么没有换脸,只是蒙版遮罩 HOT 1
- Are you still working on SimSwap++? HOT 2
- Problem ValueError with SimSwap HOT 4
- Export result of multispecific simswap colab output is not swapping faces HOT 4
- MultiSpecific.ipynb no swiping faces HOT 4
- NoneType error. HELP! HOT 1
- How was the arcface.tar generated?
- 512 vs 224 model HOT 1
- Can't get Simswap to work after upgrading PC HOT 7
- MultiScale Discriminator
- antelope.zip HOT 1
- fsmode, pix2pixhd model,ui model, projected model and pg_model folder ? HOT 1
- Error:test_video_swapsingle.py HOT 1
- Collab demo gives me "Deprecated alias error."
- Trying to install insightface==0.2.1 onnxruntime moviepy HOT 2
- Clone
- various issues on Colab
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from simswap.