yuvalnirkin / fsgan Goto Github PK

(fgan) D:\data\dev\projects>python download_fsgan_models.py
Traceback (most recent call last):
File "download_fsgan_models.py", line 8, in
from fsgan.utils.utils import download_from_url
File "D:\data\dev\projects\fsgan\utils\utils.py", line 9, in
import requests
ModuleNotFoundError: No module named 'requests'

where can i find tests directory？

I have debug this train_reenactment_attr_no_seg_v2_1.py file, but encounting many errors. Can this py file be used for training, or would you release the training code?

I would like to test, make a notebook please

Hey,

I've been trying to reproduce this tutorial, namely just this command:
python swap.py ../docs/examples/shinzo_abe.mp4 -t ../docs/examples/conan obrien.mp4 -o . --seg_remove_mouth

The swapping process seems to work fine but hangs at the end on this line.

I looked further and it seems that there is an issue somewhere in the rendering pipeline, looks like it dies here in the img_tensor.clone() operation. It looks like the video renderer doesn't even complete the processing of a single batch of frames so the completed boolean flag is never put on the _reply_queue?

If I trace the problem further back then I find that any operation applied to the tensors received from the multiprocessing _input_queue (e.g. a simple .sum()) seem to immediately kill the process, although their shape and values look fine if I print those. If I try an operation on the tensors immediately before they are put on the queue then it works fine.

I don't know much at all about pytorch.multiprocessing but I found this and this which suggest that the .put() process should stay open until at least until the .get() process has finished (and maybe until the tensors have been processed?).

I tried using .wait() and .set() with a pytorch.multiprocessing.Event() to keep the .put() process open but it didn't help (maybe I'm doing it wrong).

I am running ubuntu on a docker container.

Any thoughts?

EDIT: I removed multiprocessing from VideoRenderer and it seems to work fine now - maybe this ok as a temporary fix if rendering performance isn't such an issue?

Hi Yuval.

I'm wondering what's your pytorch and torchvision version? There are several issues related to version mismatch I found, like "axis=1" => "dim=1" in hopenet.py, and self.inplace argument redundant in img_lms_pose_transform.py.

Thanks

Good afternoon. Are there any restrictions on the length of the clips for processing, and a comparison of pre-trained models of versions 1 and 2 ,and how is it possible to improve video quality?

Hello, in the process I have a problem, after the process is completed - it runs indefinitely and the finished file weighs 44kb

=> Detecting faces in video: "source.mp4..."
100%|██████████| 250/250 [01:48<00:00,  2.29frames/s]
100%|██████████| 251/251 [00:00<00:00, 8262.21it/s]=> Extracting sequences from detections in video: "source.mp4"...
=> Cropping video sequences from video: "source.mp4"...
  3%|▎         | 8/250 [00:00<00:03, 75.44it/s]
100%|██████████| 250/250 [00:02<00:00, 124.51it/s]
=> Computing face poses for video: "source_seq00.mp4"...
100%|██████████| 2/2 [00:01<00:00,  1.90batches/s]
=> Computing face landmarks for video: "source_seq00.mp4"...
100%|██████████| 4/4 [00:01<00:00,  2.98batches/s]
=> Computing face segmentation for video: "source_seq00.mp4"...
100%|██████████| 6/6 [00:03<00:00,  1.96batches/s]  0%|          | 0/150 [00:00<?, ?frames/s]
=> Detecting faces in video: "target.mp4..."
100%|██████████| 150/150 [01:05<00:00,  2.30frames/s]
100%|██████████| 151/151 [00:00<00:00, 2776.19it/s]=> Extracting sequences from detections in video: "target.mp4"...
=> Cropping video sequences from video: "target.mp4"...
  5%|▌         | 8/150 [00:00<00:01, 77.80it/s]
100%|██████████| 150/150 [00:01<00:00, 111.29it/s]
=> Computing face poses for video: "target_seq00.mp4"...
100%|██████████| 2/2 [00:00<00:00,  2.67batches/s]
=> Computing face poses for video: "target_seq01.mp4"...
100%|██████████| 2/2 [00:00<00:00,  2.77batches/s]
=> Computing face landmarks for video: "target_seq00.mp4"...
100%|██████████| 3/3 [00:00<00:00,  3.32batches/s]
=> Computing face landmarks for video: "target_seq01.mp4"...
100%|██████████| 3/3 [00:00<00:00,  3.32batches/s]
=> Computing face segmentation for video: "target_seq00.mp4"...
100%|██████████| 4/4 [00:01<00:00,  2.11batches/s]
=> Computing face segmentation for video: "target_seq01.mp4"...
100%|██████████| 4/4 [00:01<00:00,  2.11batches/s]
=> Finetuning the reenactment generator on: "source_seq00.mp4"...
100%|██████████| 200/200 [02:55<00:00,  1.14batches/s]
=> Face swapping: "source_seq00.mp4" -> "target_seq00.mp4"...
100%|██████████| 10/10 [00:31<00:00,  3.12s/batches]
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-7-70ebda5b5352> in <module>()
     10 output_path = '/content/output.mp4'
     11 face_swapping(source_path, target_path, output_tmp_path,
---> 12               select_source, select_target, finetune)
     13 
     14 # Encode with audio and display result

5 frames
/usr/lib/python3.6/multiprocessing/connection.py in _recv(self, size, read)
    377         remaining = size
    378         while remaining > 0:
--> 379             chunk = read(handle, remaining)
    380             n = len(chunk)
    381             if n == 0:

KeyboardInterrupt: 

This is more of a question, my 11gb k80 does the performance script in 5 minutes, does anyone know if a 2080ti would do that in seconds?

Trying to test swap.py in the simplest way with two images but it crashes at the Cache input section of the FaceSwapping call function:

python swap.py ../docs/examples/source_01.png -t ../docs/examples/target_01.jpg
=> using GPU devices: 0
=> Loading face pose model: "hopenet_robust_alpha1.pth"...
=> Loading face landmarks model: "hr18_wflw_landmarks.pth"...
=> Loading face segmentation model: "celeba_unet_256_1_2_segmentation_v2.pth"...
=> Loading face reenactment model: "nfv_msrunet_256_1_2_reenactment_v2.1.pth"...
=> Loading face completion model: "ijbc_msrunet_256_1_2_inpainting_v2.pth"...
=> Loading face blending model: "ijbc_msrunet_256_1_2_blending_v2.pth"...
=> Extracting sequences from detections in video: "source_01.png"...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1659.14it/s]
Traceback (most recent call last):
File "swap.py", line 503, in
main(**vars(parser.parse_args()))
File "swap.py", line 497, in main
face_swapping(source[0], target[0], output, select_source, select_target)
File "swap.py", line 240, in call
source_cache_dir, source_seq_file_path, _ = self.cache(source_path)
File "/home/burrito/Documents/Viktoria/MIDAS/fsgan/preprocess/preprocess_video.py", line 462, in cache
self.resolution, self.crop_scale)
File "/home/burrito/Documents/Viktoria/MIDAS/fsgan/preprocess/crop_image_sequences.py", line 21, in main
raise RuntimeError('Cache file does not exist: ' + cache_path)
RuntimeError: Cache file does not exist: ../docs/examples/source_01/source_01_dsfd_seq.pkl

however I see there is a file created at that location labeled source_01_dsdf.pkl. Is there something else I should pass as arguments to swap.py for working with single images?

Hi, it seems this network is primarily for face swapping. Can it also be used for just supplying an image of a person to the network and then the network outputs a mask which is the face segmentation?

Can you help me on this error
I don't have idea what I have done wrong.
Where to put fsgan and what to put on the projects folder.
There you can see my location of files

Hello, I get this error at the end of processing swap.py. Is there a fix for this? Thank you!

Process FaceSwappingRenderer-1:
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/data/dev/projects/fsgan/utils/video_renderer.py", line 159, in run
assert frame_bgr is not None, f'Failed to read frame {i} from input video: "{self._in_vid_path}"'
AssertionError: Failed to read frame 1253 from input video: "../docs/examples/

Hello ,
I have finish some steps but now I'm stuck at python download_fsgan_models.py
I have filled the form and download the weight file but I don't have any idea where to put it and where to place download_fsgan_models.py. It says in the root directory of the fsgan repository but it doesn't work for me .
Can you explain me more in details these steps :

It show me like this :

hi again, when using the face reincarnation on v1 it only uses around 67% of one 1080ti, and i have 4 of those cards with 0 percent utilization. my question is how do i get it to use all those gpus, in turn having a shorter export time.

When I use face Reenactment the output has the skin of the src, the expressions of the target, and the head shape of the target.

what i want is for the output to have src skin, src head shape and target expressions.

I have done a video to video swapping face using google colab and it was good result.
I want to know how can I swap my single image to the video of Barack Obama for example

I'm following the steps in google colab it's going good but I want to change the source video to image .
What should I change on script calling the process :

Hi guys I’m using a K80 with 11gb vram and I’m getting a out of memory error, idk what really to do

Hello ,
I'm stuck in the step when I have to run the code download_fsgan_models.py ?
I have put this script in fsgan repository .But I don't know how should I run the code ?What should I write on anaconda navigator?
Please help me ,I'm stuck on this step already 3 weeks

When I was testing, it reported that missing hopenet_robust_alpha1.pth, but the downloaded model did not contain this model, where should I get it.

Hey, is there a plan for a Google Collab demo and/or examples/tutorials for Face Reenactment?

Thanks for the great work!
Since v2 reenactment model downloaded is nfv_msrunet_256_1_2_reenactment_v2.1.pth, while v1 version is ijbc_msrunet_256_1_2_reenactment_stepwise_v1.
Did you train v2 on ijbc? could you release that model for downloading? Thanks!

Hello ,
Thank you for this code ,I was waiting so long for it.
I'm new at this one and I don't have any idea how to start the code .
Can you give me any explanation or tuto,where to put the source image and video destination ,where to do a training then ?
Can you give me a little help on it ?

Regards

@YuvalNirkin Great work! Thanks!

Could you provide the datasets you have used for training the networks?

Hi, Thanks for the great work. Is there any documented way to run the models that are trained as per the FSGAN paper? Default options point to *_v2.pth and I am wondering is this the one as described in the paper?

When I try to pass the *_v1.pth paths it shows:

File "../../fsgan/utils/utils.py", line 179, in load_model assert arch is not None or 'arch' in checkpoint, 'Couldn\'t determine %s model architecture!' % name AssertionError: Couldn't determine face reenactment model architecture!
Please help in this regard.

Can’t find a way to add a path to my own source and target on swap.py anyone know a way?

Hi is it normal, it works great but there is no sound output

Is there a way to getTesla P100 GPU on colab other than factory resetting runtime?I tried resetting runtime many times but I am always getting Tesla K80 .I am getting CUDA out of memory error for some videos.
I tried reducing batch_size and segment_batch_size but the error persists.
Please look into this matter.

I am getting this error from here:
File "../../fsgan/preprocess/preprocess_video.py", line 20, in <module> from face_detection_dsfd.face_detector import FaceDetector
I am wondering whether I need to install this package? Please suggest .

When I try to call the swap inference script writing this command

python swap.py SOURCE -t TARGET -o OUTPUT [FLAGS]

It show me this

Hello @YuvalNirkin
when I tested fsgan model on custom images using face_swap_images2images.py, the results look like this.
Sequence of image is : source img -> target img -> result img
Please let me know Is there any solution to resolve this issue.

the program runs fine, everything finishes, but i cant find the final face-swaped video, i know its not the one located in the conon_obrien folder, any help would be appreciated

Thanks for releasing the code!
A Colab notebook is a great way to help people get started. I'd really appreciate it as I find it much easier to get familiar with a new project like that.
Regards

Running a colab demo

when i run to swap a face it gives an error

i am using python3 in linux

cd fsgan/inference
python swap.py ../docs/examples/shinzo_abe.mp4 -t ../docs/examples/conan_obrien.mp4 -o . --finetune --finetune_save --seg_remove_mouth

=> using GPU devices: 0
=> Loading face pose model: "hopenet_robust_alpha1.pth"...
=> Loading face landmarks model: "hr18_wflw_landmarks.pth"...
=> Loading face segmentation model: "celeba_unet_256_1_2_segmentation_v2.pth"...
=> Loading face reenactment model: "nfv_msrunet_256_1_2_reenactment_v2.1.pth"...
=> Loading face completion model: "ijbc_msrunet_256_1_2_inpainting_v2.pth"...
=> Loading face blending model: "ijbc_msrunet_256_1_2_blending_v2.pth"...
=> Cropping video sequences from video: "conan_obrien.mp4"...
Could not find encoder for codec id 27: Encoder not found
100% 150/150 [00:01<00:00, 84.60it/s]
=> Computing face poses for video: "conan_obrien_seq00.mp4"...
Traceback (most recent call last):
  File "/content/swap.py", line 498, in <module>
    main(**vars(parser.parse_args()))
  File "/content/swap.py", line 492, in main
    face_swapping(source[0], target[0], output, select_source, select_target)
  File "/content/swap.py", line 237, in __call__
    source_cache_dir, source_seq_file_path, _ = self.cache(source_path)
  File "/content/fsgan/preprocess/preprocess_video.py", line 463, in cache
    self.process_pose(input_path, output_dir, seq_file_path)
  File "/content/fsgan/preprocess/preprocess_video.py", line 242, in process_pose
    in_vid = VideoInferenceDataset(curr_vid_path, transform=img_transforms)
  File "/content/fsgan/datasets/video_inference_dataset.py", line 29, in __init__
    self.width, self.height, self.total_frames, self.fps = get_video_info(vid_path)
  File "/content/fsgan/utils/video_utils.py", line 223, in get_video_info
    return get_media_info(vid_path)
  File "/content/fsgan/utils/video_utils.py", line 206, in get_media_info
    probe = ffmpeg.probe(media_path)
  File "/usr/local/lib/python3.6/dist-packages/ffmpeg/_probe.py", line 23, in probe
    raise Error('ffprobe', out, err)
ffmpeg._run.Error: ffprobe error (see stderr output for detail)

Hello.
the description says about windows, but the guide is only for Linux. When will windows guide appear?
Thanks

Traceback (most recent call last):
File "/home/hsl/Project/Face-swap/fsgan/swap.py", line 501, in
main(parser.parse_args())
File "/home/hsl/Project/Face-swap/fsgan/swap.py", line 490, in main
blending_model=blending_model, criterion_id=criterion_id, min_radius=min_radius, output_crop=output_crop)
File "/home/hsl/Project/Face-swap/fsgan/swap.py", line 131, in init
self.Gr, checkpoint = load_model(reenactment_model, 'face reenactment', self.device, return_checkpoint=True)
File "/home/hsl/Project/Face-swap/fsgan/fsgan/utils/utils.py", line 166, in load_model
checkpoint = torch.load(model_path, map_location=lambda storage, loc: storage)
File "/home/hsl/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 368, in load
return _load(f, map_location, pickle_module)
File "/home/hsl/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 549, in _load
deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 1880922 more bytes. The file might be corrupted.

I meet this problem and have no idea to address it

hi i get this when trying to faceswap, reenact works fine

Process FaceSwappingRenderer-1:
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/data/dev/projects/fsgan/utils/video_renderer.py", line 159, in run
assert frame_bgr is not None, f'Failed to read frame {i} from input video: "{self._in_vid_path}"'
AssertionError: Failed to read frame 592 from input video: "/data/dev/projects/videos/conan_obrien.mp4"

Can you make a Colab of V1?!

I can't use the other models :/

I was trying to run it on colab exactly as given in your guideline and did not change a thing. This is the error i got in the very last cell. I know this not the issue in the repo but I just wanted to know if you were using extended DRAM in colab or any pro version or any other solution you might know to this error.

Thank You

RuntimeError: CUDA out of memory. Tried to allocate 1014.00 MiB (GPU 0; 11.17 GiB total capacity; 9.56 GiB already allocated; 989.81 MiB free; 9.90 GiB reserved in total by PyTorch)

I'm following the steps from Face Swapping Inference
I'm stuck on the second step trying to call the swap inference script
When I run python swap.py it show me this error :

Hi, thanks for your great work.
Conventionally, pix2pix discriminators are conditioned on generator inputs, right? It seems your training code in version 1 only takes generated image as input to D. What is your insight and did you try conventional conditional-GAN?

I want to swap images in stead of videos can you please tell me how can I do that. I tried to input images but it gives error.

RuntimeError: Cache file does not exist: /content/data/source/source_dsfd_seq.pkl

My GPU is TITANV, 12GB.
i run the command "python swap.py ../docs/examples/shinzo_abe.mp4 -t ../docs/examples/conan_obrien.mp4 -o . --finetune --finetune_save --seg_remove_mouth"
the error info:
File "/git/face_detection_dsfd/face_ssd_infer.py", line 47, in forward
return torch.cat((x1_1, x2_1, x3_1), 1)
RuntimeError: CUDA out of memory. Tried to allocate 1.98 GiB (GPU 0; 11.78 GiB total capacity; 8.08 GiB already allocated; 1.22 GiB free; 1.30 GiB cached)

I find that it processes shizao_abe.mp4 successfully, because its resolution is 1280 x720. But conan_obrien.mp4 is failed, because its resolution 1920x1080.
Does the resolution influence the detection DRAM?

Don’t know if it’s meant to happen, but when it outputs the video is cropped to the face

I'm trying to get reenact working in Google Colab. I was installing requirements and eventually got stuck on this one. I have fsgan in /content/projects/fsgan. Any help is much appreciated.

The error clearly indicating that it's not finding fsgan.

Traceback (most recent call last):
  File "/content/projects/fsgan/inference/reenact.py", line 23, in <module>
    from fsgan.preprocess.preprocess_video import VideoProcessBase, base_parser
ModuleNotFoundError: No module named 'fsgan'

I'm using it like this: python reenact.py <source image> -t <target video> -o <output directory> which I'm guessing is related to why it's not working.

I'll keep referring to the existing Colab and see if I can't figure it out.