python: can't open file 'test_video_swapsingle.py': [Errno 2] No such file or directory
how to fix

Thanks!
And did you tried unet structure in you'r experiments, which may be good for reversing attribute, especially background.

Hi!
The work is great!
In your paper, you train one batch for image pairs with the same identity and another
batch for image pairs with different identities alternately. what is the batch size when you train? And have you tried a different batchsize.
Thank you

Please Index a Colab notebook here

Love this project, and the demo project/colab works fine if I don't touch the demo image and the demo video, but as soon as I change either one to test it with my own samples I get an error. Can someone help tell me what I'm doing wrong, or missing?

Here's the output it gives me when it fails.

`AttributeError Traceback (most recent call last)
in ()
23 # img_a = Image.open(pic_a).convert('RGB')
24 img_a_whole = cv2.imread(pic_a)
---> 25 img_a_align_crop, _ = app.get(img_a_whole,crop_size)
26 img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB))
27 img_a = transformer_Arcface(img_a_align_crop_pil)

1 frames
/usr/local/lib/python3.7/dist-packages/insightface/model_zoo/scrfd.py in detect(self, img, threshold, input_size, max_num, metric)
202 input_size = self.input_size if input_size is None else input_size
203
--> 204 im_ratio = float(img.shape[0]) / img.shape[1]
205 model_ratio = float(input_size[1]) / input_size[0]
206 if im_ratio>model_ratio:

AttributeError: 'NoneType' object has no attribute 'shape'`

is there a patch for the colab to fix the error when the video has no audio?

Hi!
The work is incredible! But I'm interested in the dataset -- what is the number of images that you trained on? And how much did it take in terms of time (and on which GPUs)?
Another question is if you know where to take this dataset, because as for now it is not available using the direct link https://www.robots.ox.ac.uk/~vgg/data/vgg_face2

Traceback (most recent call last):
File "test_video_swapsingle.py", line 34, in
opt = TestOptions().parse()
File "C:\Program Files\SimSwap\SimSwap-main\options\base_options.py", line 84, in parse
torch.cuda.set_device(self.opt.gpu_ids[0])
File "C:\Users[ME]\anaconda3\envs\simswap\lib\site-packages\torch\cuda_init_.py", line 261, in set_device
torch._C._cuda_setDevice(device)
AttributeError: module 'torch._C' has no attribute '_cuda_setDevice'

How do I fix this, at first I thought I might have the wrong pytorch, but I downloaded it from simswap github https://github.com/neuralchen/SimSwap/blob/main/docs/guidance/preparation.md

Hello, I've sent some videos to a couple of friends, and none of them could listen to the audio (iphones and macbooks), there was no audio at all.

I had no problem listening to them on android or on windows.

Replacing the audio from the output file using ffmepg solved the problem.

when will training support?

Thank you for excellent work! when will you release the training code?

I've seen other code that's been added to fsgan to attempt to inject emotion.
BigBey/emotion_transfer@24637fc

because the architecture is quite simple - presumably - you'd just need to do some neural net surgery on the ID injection layer?

if you have any line numbers to / blended networks to play with.....

Also - I see the pix2pix model is part of the architecture - were there any efforts to tack on stylegan2 into the mix?

First of all, THANK YOU for working on this wonderful project! ❤
I found this project few days ago, I'm pretty much fall in love with it right away.

After trying the latest 2 features added:

1. Remove Watermark
2. Swap SPECIFIC Face

It inspired me as a simple user, how can I help the project grow with my own ideas & skills.

Suggestion / Ideas:

1. Replace 'SPECIFIC' Multiple Faces at once
2. GUI - Easy to use basic user-interface

1. If it's possible to go a bit farther with the "Replace Specific Face" to multiple?
For example, good reference is something like ReFace App:
First, it analyze the video find ALL the faces (I guess it snapshot and crop them as best as it can consider if face is close to edge or something it gives extra space on the other sides),
Then it allow the user to choose / select which one of the founded faces to swap: just a specific face or every single one of them. (total freedom of selection to the user)

2. Related to above suggestion, if possible it will be AMAZING :
Can you make a super-simplistic (basic as possible for start) GUI so we run 1 command it will open the GUI, this will help the user with some preview and controls such as:

• Having all the features (current and future optional usage) working in one place.
• Selecting the Main Destination Video or Image
• Run video or image analyze (will recognize and crop each discovered face)
• Align + Crop each founded face (so the user could SEE + Replace)
• When analyze finished: Show 1 thumbnail of each of the cropped founded faces
• Select / Drag & Drop to the SPECIFIC faces (thumbnails) with the wanted 1 or multiple Source image/s (the faces you want to use instead).
• Run the Final Render-Process to Analyze, Align, Crop selected Source Images and merge all together.
• When done, it can either play the final video inside the GUI as preview
• If you like it, you can keep it or REDO change the SPECIFIC faces again, or anything else

The most helpful thing will be to SEE the recognized face of the current video / image so the users can CLICK the thumbnail of FACE-1 and swap to (either drag and drop or just load from our PC) the face we want, same for FACE-2, FACE-6 or Remove their selection, so it's easy to test results in one place which is the main GUI.

In case of multiple faces, if you don't swap one of the SPECIFIC faces, it will just leave them as the original video / image and do it's swapping magic.

❤ I can help with the GUI + GUI Design:
I'm aware a GUI could be different, but it's a very inviting user-friendly for sure.

If you need help with the GUI design + buttons, I'll be happy to help using my skills:
I can design the basic GUI mockup for reference and the buttons (if needed), maybe you just want to draw whatever default buttons uses for minimalistic GUI which is fine and maybe faster to produce. again, I'm not a programmer I'm an animator + game/software designer so I look at it from other angle.

THE MAIN IDEA
Is to keep it simple for the user, but also start re-build upcoming features based on a specific Base-GUI so the user may see immediately new buttons or options when update.
If you will like my idea I'll be happy to help with the GUI graphics design: make a rough Mockup + Design the buttons.

• Of course I have SO MANY IDEAS for extra features... but let's start with the basics first ;)

Thanks ahead and sorry about my bad English, I hope that you can understand most of what I wrote.

I've installed on Windows using Anaconda and when trying to run my first demo using: "Face swapping for video" command with the default demo files.

I get these errors:

Can somebody please guide me step-by-step how to fix that?
Thanks ahead!

Sorry if my question sound too rude to ask:
Is it possible to remove the watermark with an addition parameter?

In case the author are not interested allowing to remove the watermark I will respect that of course.
Just to be clear I didn't mean to be rude but wondering if possible.

Thanks ahead and please keep up the good work! :)

Hi, great work !

I am trying on my own images, and the result changes a lot depending on the cropping conditions of the face. What backend are you using to align your faces ? I use face-alignment to detect the face, crop the face centered on the nose and rotated along the axis eye_average->mouth. Still I don't know the extent of the cropping. Can you help me with that ?

Thanks a lot in advance !

Guys, you might wanna fix some typos with some of the python files. Check the screenshot for more details.

Thanks for your wonderful work and opening the source codes. I have implemented the training codes. To make the background stable, I predict the mask and merge the generated image with the target image as

how to make video faceswap as your example

I would like to know if it is possible to adjust the force of the face swap.

for example

force = 0
show original face
force = 0.5
original and new face mixed
force = 1
show the new face

It's a great project. I follow the instructions to configure the environment. When the code runs to netArc_checkpoint=torch.load(netArc_checkpoint) an error is reported.
..torch/serialization.py line 702, in _legacy_load
result = unpickler.load()
AttributeError: Can't get attribute 'ArcMarginModel' on <module 'models' from 'SimSwap-main/models/init.py'>
How to solve it? Is the model incorrect？

Hi, thanks for releasing the amazing work!

I'm trying to reproduce the training part, and I borrowed a lot code from fs_model.py

However, in another issues, #27 , you recommend to use "opt.model == 'pix2pixHD" to reprodeuce the base performance.

I'm wondering how are these two models used?

Btw, when I'm trying to train the model by using fs_model.py, the Gradient Penalty loss gets very high, even it is assigned by λ=1e-5, do you have any idea about this?

There's a inference code for inference without crop?

Currently this code uses a single image to transfer face look a like

I want to go frame by frame of input video and drive the target face with source motions while maintaining style / colours.

maybe not so simple

I'm trying to test the trained model using test_one_image.py as instructed in the README, but am running into the following errors. I'm using Python 3.7.10, Pytorch 1.8.1+cu101 , torchvision 0.9.1+cu101, and torchtext 0.9.1. What versions of all the libraries were used to train the model? And how do I forward port the model?

/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.parallel.data_parallel.DataParallel' has changed. Saved a reverse patch to DataParallel.patch. Run `patch -p0 < DataParallel.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.conv.Conv2d' has changed. Saved a reverse patch to Conv2d.patch. Run `patch -p0 < Conv2d.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.batchnorm.BatchNorm2d' has changed. Saved a reverse patch to BatchNorm2d.patch. Run `patch -p0 < BatchNorm2d.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.activation.PReLU' has changed. Saved a reverse patch to PReLU.patch. Run `patch -p0 < PReLU.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.pooling.MaxPool2d' has changed. Saved a reverse patch to MaxPool2d.patch. Run `patch -p0 < MaxPool2d.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.container.Sequential' has changed. Saved a reverse patch to Sequential.patch. Run `patch -p0 < Sequential.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.pooling.AdaptiveAvgPool2d' has changed. Saved a reverse patch to AdaptiveAvgPool2d.patch. Run `patch -p0 < AdaptiveAvgPool2d.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. Saved a reverse patch to Linear.patch. Run `patch -p0 < Linear.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.activation.Sigmoid' has changed. Saved a reverse patch to Sigmoid.patch. Run `patch -p0 < Sigmoid.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.dropout.Dropout' has changed. Saved a reverse patch to Dropout.patch. Run `patch -p0 < Dropout.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.batchnorm.BatchNorm1d' has changed. Saved a reverse patch to BatchNorm1d.patch. Run `patch -p0 < BatchNorm1d.patch` to revert your changes.
  warnings.warn(msg, SourceChangeWarning)
Traceback (most recent call last):
  File "test_one_image.py", line 35, in <module>
    model = create_model(opt)
  File "/content/SimSwap/models/models.py", line 18, in create_model
    model.initialize(opt)
  File "/content/SimSwap/models/fs_model.py", line 64, in initialize
    netArc_checkpoint = torch.load(netArc_checkpoint)
  File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 585, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 772, in _legacy_load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 1918589 more bytes. The file might be corrupted.

C:\ProgramData\Miniconda3\envs\simswap\lib\site-packages\torch\serialization.py:658: SourceChangeWarning: source code of class 'torch.nn.parallel.data_parallel.DataParallel' has changed. Tried to save a patch, but couldn't create a writable file DataParallel.patch. Make sure it doesn't exist and your working directory is writable.

when trying to run the last bit of code I get this error

After I've downloaded the latest version I'm getting an error when I try to run the mutispecific command for video:

set det-size: (640, 640)
(142, 366, 4)
  0%|                                                                                                                                                                                                                | 0/56 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "test_video_swap_multispecific.py", line 93, in <module>
    model, app, opt.output_path,temp_results_dir=opt.temp_path,no_simswaplogo=opt.no_simswaplogo)
  File "Z:\SimSwap\util\videoswap_multispecific.py", line 83, in video_swap
    min_indexs = np.argmin(id_compare_values_array,axis=0)
  File "<__array_function__ internals>", line 6, in argmin
  File "Z:\anaconda3\envs\simswap\lib\site-packages\numpy\core\fromnumeric.py", line 1269, in argmin
    return _wrapfunc(a, 'argmin', axis=axis, out=out)
  File "Z:\anaconda3\envs\simswap\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: attempt to get argmin of an empty sequence

What am I doing wrong? and how can I fix it?
it worked fine on the version before for few days ago...

UPDATE: (it works now)
It's funny because it was my suggestion (Thanks for adding this!) the file names are now on the same directory + named to SRC_01.jpg and DST_02.jpg - GREAT!
But I believe it's supposed to be the opposite:

• SRC = The source image you want to swap to the destination.
• DST = The destination image which should be swap in to. (to final result)

I'm now trying the multi-specific for the images, still getting issues with it... if I won't catch my mistake I may post it so I can get some help, thanks and please keep up the good work!

Hi @NNNNAI @AceSix :
Many thanks for your excellent works. However, I can't find training code .Can you provide this code? Thank you very much

UnpicklingError Traceback (most recent call last)
in ()
13
14 torch.nn.Module.dump_patches = True
---> 15 model = create_model(opt)
16 model.eval()
17

3 frames
/usr/local/lib/python3.7/dist-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
775 "functionality.")
776
--> 777 magic_number = pickle_module.load(f, **pickle_load_args)
778 if magic_number != MAGIC_NUMBER:
779 raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: invalid load key, '<'.

How would this be fixed?

Hey guys! thank you for the amazing paper!
While trying to implement a training script for the model I got the following question:
What type of input and output normalization did you use? Am I correct that it was [0..1] for the input image, and tanh ([-1..1]) as output? What was the input for the Discriminator: [-1..1] or [0..1]?

I will be grateful for the answer.

Can you give me a rough time? Like within 1 week or 1 month, etc.
Thanks a lot!

Does it will run on macbook pro m1 8GB ram?

First of all, great work. Are there parameters that can keep more of the details from the source image, or is this something that needs to be trained on? For example, sometimes there might be key details missing from the eyes, or maybe there are other features(piercings, tattoos, moles, and so on for example) that I may want to keep. Thanks!

It would be nice if someone put together a Google Colab for this project.

Like this DFL project:
https://colab.research.google.com/github/chervonij/DFL-Colab/blob/master/DFL_Colab.ipynb

This would be very convenient for those who do not have powerful GPUs.

根据Usage所提供的命令运行出现如下错误。
运行的命令：
Face swapping for video
Swap only one face within the video(the one with highest confidence by face detection).
python test_video_swapsingle.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --video_path ./demo_file/mutil_people_1080p.mp4 --output_path ./output/mutil_test_swapsingle.mp4 --temp_path ./temp_results
出现的错误：
Traceback (most recent call last):
File "test_video_swapsingle.py", line 44, in
app = Face_detect_crop(name='antelope', root='./insightface_func/models')
File "E:\BaiduNetdiskDownload\SimSwap-main\insightface_func\face_detect_crop_single.py", line 39, in init
assert 'detection' in self.models
AssertionError

First of all, great work!

I wanted to ask you how you evaluated against "Fine-grained head pose estimation without keypoints" for face pose correctness, because the official implementation of the Hopenet works on Python 2.7 with torch 0.4, incompatible with your environment and most modern systems. I was wondering if you used a different or more modern implementation of it.

Thank you!

The arcface_checkpoint.tar downloaded from Google Drive failed to decompressed after being verified by Win and Linux.

Excuse me, is the file damaged?

when I use an custom image I get this

-------------- End ----------------
input mean and std: 127.5 127.5
find model: ./insightface_func/models/antelope/glintr100.onnx recognition
find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection
set det-size: (640, 640)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-c61bb4ca883a> in <module>()
     23 # img_a = Image.open(pic_a).convert('RGB')
     24 img_a_whole = cv2.imread(pic_a)
---> 25 img_a_align_crop, _ = app.get(img_a_whole,crop_size)
     26 img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB))
     27 img_a = transformer_Arcface(img_a_align_crop_pil)

TypeError: cannot unpack non-iterable NoneType object

Since I got it to work on my GForce 1050GTX / 2GB , at least for videos not longer than ~ 16 frames, before the GPU runs out of memory I wonder if there is also a limitation for using a 8 GB GPU ?

I had the same problem using Wav2Lip, but it could be solved by setting the chunk size to 1.

Would it (theoretically) be possible to process videos in SimSwap in smaller parts or chunks by releasing GPU memory every 15 frames ?