anothermartz / easy-wav2lip Goto Github PK

Here is the error message:

Processing trump.mp4 using trump-aaron.wav for audio
imports loaded!
Processing failed! :( see line above 👆
Consider searching the issues tab on the github:
https://github.com/anothermartz/Easy-Wav2Lip/issues

No details about how it failed. Any idea?

Traceb

raceback (most recent call last):
  File "/Easy-Wav2Lip/inference.py", line 786, in <module>
    main()
  File "/Easy-Wav2Lip/inference.py", line 734, in main
    f[y1:y2, x1:x2] = p
ValueError: could not broadcast input array from shape (620,555,3) into shape (74,166,3)

i only run the inference.py and do not change the code

Windows 10

Hello.

When I try to install it by the Easy-Wav2Lip_v8.1.bat, I have some troubles.

It shows:

Welcome to Easy-Wav2Lip
Easy-Wav2lip apears to not be installed correctly, reinstall?
You will need around 2GB of free space.
Press any key to continue . . .
saving the large downloaded files
Easy-Wav2Lip\checkpoints\mobilenet.pth
Easy-Wav2Lip\checkpoints\README.md
2 File(s) copied
Cloning into 'Easy-Wav2Lip'...
remote: Enumerating objects: 1448, done.
remote: Counting objects: 100% (725/725), done.
remote: Compressing objects: 100% (366/366), done.
remote: Total 1448 (delta 465), reused 571 (delta 355), pack-reused 723
Receiving objects: 100% (1448/1448), 16.13 MiB | 12.20 MiB/s, done.
Resolving deltas: 100% (689/689), done.
temp\checkpoints\mobilenet.pth
temp\checkpoints\README.md
2 File(s) copied
'pip' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "C:\AI\Easy-Wav2Lip\Easy-Wav2Lip\install.py", line 6, in
from IPython.display import clear_output
ModuleNotFoundError: No module named 'IPython'
Easy-Wav2lip apears to not be installed correctly, reinstall?
You will need around 2GB of free space.
Press any key to continue . . .

If I press any key, it keeps looping, and say:

Easy-Wav2lip apears to not be installed correctly, reinstall?
You will need around 2GB of free space.
Press any key to continue . . .

Is that the issue of "ModuleNotFoundError: No module named 'IPython'"?

If so, how to fix it?

Thank you!

Hi, I have another issue.

Some of the files I process, it will cause error. IndexError: list index out of range

Then it will stop.

May I know how to fix that?

Thank you.

Below is the detail.

Processing slience-enhance.mp4 using news03-a.wav for audio
imports loaded!
analysing audio...
826 frames to process
Using face detection data from last input
mask size: 2.5, feathering: 3
Loading gfpgan
Starting...
Processing Wav2Lip: 56%|██████████████████████▉ | 463/826 [08:50<06:55, 1.15s/it]
Traceback (most recent call last):
File "C:\AI\Easy-Wav2Lip\inference.py", line 694, in
main()
File "C:\AI\Easy-Wav2Lip\inference.py", line 593, in main
for i, (img_batch, mel_batch, frames, coords) in enumerate(tqdm(
File "C:\Users\wesley\AppData\Local\Programs\Python\Python310\lib\site-packages\tqdm\std.py", line 1178, in iter
for obj in iterable:
File "C:\AI\Easy-Wav2Lip\inference.py", line 469, in datagen
face, coords = face_det_results[idx].copy()
IndexError: list index out of range
Processing failed! :( see line above 👆
Processing failed

how can I download the .pth and .pkl checkpoint files?

Traceback (most recent call last):
File "c:\maker\EasyWav2Lip\inference.py", line 117, in
with open(os.path.join('checkpoints','predictor.pkl'), 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints\predictor.pkl'

I have problem with running it on Kaggle. I thought if we can run it on Colab there is possible way to run it on Kaggle. Ofc I changed folders and tryed a lot of solutions with installing libxcb - did not work.

Processing 35685012a_15-54-52.mp4 using 4.wav for audio
imports loaded!
analysing audio...
500 frames to process
Using face detection data from last input
mask size: 1.5, feathering: 1
Loading gfpgan
Starting...
qt.qpa.xcb: could not connect to display
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/opt/conda/lib/python3.10/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.

Processing failed! :( see line above 👆
Consider searching the issues tab on the github:
https://github.com/anothermartz/Easy-Wav2Lip/issues

Originally posted by @KeraiSensei in #37 (comment)

Traceback (most recent call last):
File "D:\1\Easy-Wav2Lip\run.py", line 81, in
in_width, in_height, in_fps, in_length = get_video_details(video_file)
File "D:\1\Easy-Wav2Lip\easy_functions.py", line 33, in get_video_details
length = float(info['format']['duration'])
KeyError: 'duration'

SystemExit: Step 1 has not been run in this instance! Please run step 1 each time you disconnect from a runtime.

Im launched step 1 but i still have this problem

dilb has not been installed in linux system, what is this? I have cmake and gcc installed for linux, but dilb still can't be installed:
File "", line 130, in run
File "", line 170, in build_extension
File "/data/anaconda3/envs/easywav2lip/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess. CalledProcessError: Command '['cmake', '--build', '. ', '--config', 'Release', '--', '-j64']' returned non-zero exit status 2.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for dlib
Failed to build dlib
ERROR: Could not build wheels for dlib, which is required to install pyproject. toml-based projects

During the synthesis of video, I found that if the original video was not in the silent state, there would be a problem of mouth mismatch, but in the silent state, the effect is much better. I would like to ask how to match the mouth shape in the original video speaking state, or directly generate a silent video (I found that the effect of Wav2Lip trying to silence is not good), thank you very much

Sorry for the persistence, I’ve been waiting for the second day for the problem to be corrected. I saw your previous messages, you promised to fix it as soon as you get to the computer. I hope everything is okay with you. Looking forward to the update!

Thank you very much!

Always everything ran fine: but now it gives this error. tell me how to solve the problem? FileNotFoundError: [Errno 2] No such file or directory: 'last_file.txt'

Error logs:

Traceback (most recent call last):
File "/content/Easy-Wav2Lip/run.py", line 80, in
in_width, in_height, in_fps, in_length = get_video_details(video_file)
File "/content/Easy-Wav2Lip/easy_functions.py", line 19, in get_video_details
info = json.loads(result.stdout)
File "/usr/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

when ssh excute command: python install.py
display these errors,but '/home/ps/miniconda3/envs/wav2lip/lib/python3.11/site-packages/degradations.py'
has this file'degradations.py' :
Traceback (most recent call last):
File "/home/ps/miniconda3/envs/wav2lip/lib/python3.11/shutil.py", line 853, in move
os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: 'degradations.py' -> '/home/ps/miniconda3/envs/wav2lip/lib/python3.11/site-packages/degradations.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Easy-Wav2Lip/install.py", line 38, in
move_and_replace_file_to_basicsr(file_to_replace)
File "/Easy-Wav2Lip/install.py", line 31, in move_and_replace_file_to_basicsr
shutil.move(file_name, destination)
File "/home/ps/miniconda3/envs/wav2lip/lib/python3.11/shutil.py", line 873, in move
copy_function(src, real_dst)
File "/home/ps/miniconda3/envs/wav2lip/lib/python3.11/shutil.py", line 448, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/home/ps/miniconda3/envs/wav2lip/lib/python3.11/shutil.py", line 256, in copyfile
with open(src, 'rb') as fsrc:
^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'degradations.py'

Hello, I'm encountering this problem on 3 machines with different Google Accounts:

As I understand, there might be an issue in the code due to some recent libraries update which now cannot be handled right in the code. (I might be wrong)

Could someone please help me with a fix?

1. Suggesiton: I think this project (the bat file) could benefit from the creation of a VENV, creating a venv will allow the libraries to not compete with other python libraries of your base python env. The bat file should check if the venv exist and work on it, if does not exist then create it.

2. Question 1: You mentioned using bing ai to help you create this, My experience is that bign ai tend to forget the context of the discussion so you cant have a long conversation with it, how did you manage? Maybe it was just small conversations at once?

3. Question 2: You mention cog wav2Lip , I think i had tried to use it once, but I was nevel able to make it work, proof:
   

There is always an error about librosa or NUMPY versiosn, I have no idea what to fix, when I try to install lower versions of numpy it produces some error, if I leave it it does not run..
I would like to install that versions you mentioned just to try it (on build on it possibly)
can you:
- help me install it, tell me which versions of python and which versions of the libraires you used to make it work in local, if you ever try?
- Tell me which files I need from that repo (copy paste into another working repo) so I can try it easily and make it work without having to install the whole cog repo? Where do I put the models : "Visual quality"' and "Lipsync expert"? do I put them inside checkpoints folder aswell?
edit: I made the cog wav2lip work.

3. You state that you had no python knowledge prior, but you had coding knowledge on other langauegs right? because otherwise that woudl be very surprising, a non coder could not do this even with gpt help.
   Cheers

What is the license? Can it be used for commercial purposes?

i think it may caused by ffprobe static package is not a support format for ubuntu.
i can see https://evermeet.cx/ffmpeg/ is for Mac OS system, not for linux

so, i want know how can i fix this, can u provide ffprobe static package for ubuntu， thanks .

Please where can i download the mouth_detector.pkl file and other checkpoints needed, i have searched all the internet and couldn't find one

thanks for your help

python install.py = this the error:

ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

Any idea how to fix that?

ERROR: Ignored the following versions that require a different python version: 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10
ERROR: Could not find a version that satisfies the requirement tb-nightly (from basicsr) (from versions: none)

opening GUI
Saving config
starting Easy-Wav2Lip...
Traceback (most recent call last):
File "F:\Wav2Lip-8.3\Easy-Wav2Lip-8.3_release\run.py", line 5, in
from easy_functions import (format_time,
File "F:\Wav2Lip-8.3\Easy-Wav2Lip-8.3_release\easy_functions.py", line 5, in
import dlib
ModuleNotFoundError: No module named 'dlib'
opening GUI

Hello, today Wav2Lip has bug.
After 1 step is done, 2 step inform about not finished 1 step (red button) when is already done...
Last months everything was works so well :) So i decided to put this information there to help resolve this problem.
Thanks!

raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'complex'.
np.complex was a deprecated alias for the builtin complex. To avoid this error in existing code, use complex by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.complex128 here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'complex_'?

how to use start wav2lip in local

where to load it?

Trying to use that on C:\ and D:\ (even on root) but the error is the same:

Welcome to Easy-Wav2Lip!
Create folders "Easy-Wav2Lip" and "Easy-Wav2Lip-venv" here and install Easy-Wav2Lip ?
You will need around 7GB of free space and consider the time it will take you to download that on your connection.
There may be freezes in the cmd for several minutes at a time, this is normal.
Press any key to continue . . .
Proceeding with install
Creating virtual environment...
\Windows was unexpected at this time.

I have over 100GB free on each drive (both are SSD)

Any chance for a portable version pre-installed to grab all-together?

Thanks for this. It is really great. I ran the colab and it was very good. I tried the local install on a Windows 10 machine with RTX 2060 12GB and. After filling in the config.ini I am getting a no module named torch error.

Dear author,

This is a really nice project that generates much faster than wav2lip,

Now to generate a 30 second video, it only takes about 30 seconds,

My machine configuration: two 3090 graphics cards

Very nice, thank you very much.

I have a question I need to ask

1. What method do you use to improve the generation speed of easy-wav2lip?

2. Is it possible to make easy-wav2lip generation faster? Because I want to be a real time digital person, I needed a faster rendering speed.

Looking forward to your reply.

imports loaded!
Traceback (most recent call last):
File "/home/work/anaconda3/envs/dvh/lib/python3.10/configparser.py", line 790, in get
value = d[option]
File "/home/work/anaconda3/envs/dvh/lib/python3.10/collections/init.py", line 986, in getitem
return self.missing(key) # support subclasses that define missing
File "/home/work/anaconda3/envs/dvh/lib/python3.10/collections/init.py", line 978, in missing
raise KeyError(key)
KeyError: 'preview_window'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/workspace/dvh/Easy-Wav2Lip/inference.py", line 256, in
preview_window = config.get('OPTIONS', 'preview_window')
File "/home/work/anaconda3/envs/dvh/lib/python3.10/configparser.py", line 793, in get
raise NoOptionError(option, section)
configparser.NoOptionError: No option 'preview_window' in section: 'OPTIONS'
Processing failed! :( see line above 👆
Consider searching the issues tab on the github:
https://github.com/anothermartz/Easy-Wav2Lip/issues

Hi @anothermartz ,

Thank you for your effort and its a very inspiring repo and project !

I am recently looking at Wav2Lip & GFP GAN as well, by looking at your "Mask" function, I am wondering if we can only go through GFP enhancement on the lip / mouth location only, instead of the whole face in order to save inference time?

Please advise.

Cheers!

Cat

Greetings!

I am trying to run the Colab version of the code. When I run Step 1 it finishes in about 1 and a half minutes but then when I run Step 2 I get a message saying the following:

An exception has occurred, use %tb to see the full traceback.

SystemExit: Step 1 has not been run in this instance! Please run step 1 each time you disconnect from a runtime.
/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:3561: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

Could you please help me fix this? I haven't changed anything in the code, I am just running it as is.

Thank you very much! Your code is great!

install.py throws the following error when running on Apple Silicon:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. 
If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') 
to map your storages to the CPU.

The solution was to change line 83 of easy_functions.py to explicitly reference the device value stored in the device variable:

--- a/easy_functions.py
+++ b/easy_functions.py
@@ -80,7 +80,7 @@ def format_time(seconds):

 def _load(checkpoint_path):
     if device != "cpu":
-        checkpoint = torch.load(checkpoint_path)
+        checkpoint = torch.load(checkpoint_path, map_location=torch.device(device))
     else:
         checkpoint = torch.load(
             checkpoint_path, map_location=lambda storage, loc: storage

I'm not submitting a PR bc I can't be bothered to fork this repo when I don't even know if it will work past the install. Leaving this here for anyone else who's trying to run this on a Mac.

Step 1 Cell ran successfully. But in step 2, the following error occurs;

An exception has occurred, use %tb to see the full traceback.

SystemExit: Step 1 has not been run in this instance! Please run step 1 each time you disconnect from a runtime.
/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:3561: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

installed.txt file ???

My program graphics card usage is relatively low, and I only use less than 4GB of 12GB of graphics memory. What parameters do I want to use to improve my usage and speed.

i use colab to generate vedio, my input vedio is 1080 X 1920, but out vedio is 540 X 960，i want get a config to change the output vedio file resolution, hope it can output 1080 X 1920 vedio, it will be much clear。

Hello, please tell me, I’m new to the topic of deepfakes, I’ve been figuring it out for a week now, I’ve made some progress, I found out that a new Easy-Wav2Lip has been released and I want to ask, is it possible to use this program on a MacBook? If not, what software do I need to use Easy-Wav2Lip, please give a more detailed answer

Thanks for your great contribution

How to update device from cuda into cpu.
reproduced :
edit inference.py and easy_functions.py
device = 'cuda'
into
device = 'cpu' or device = ''
while run python run.py
keep shown

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Please let me know if something missing.
Thank you a lot

Can this repo be deployed on our PC?

Hi,

I just found this project and the repository at https://github.com/instant-high/wav2lip-onnx-HQ. I think that combining both may make this even faster.

I ran some quick tests using the ONNX model from this repository (with the help of ChatGPT), and it seems I get about 15%-20% faster generation times. However, I've never implemented something like this before, so I might be doing something wrong.

Your Discord link seems to be down, so I couldn't contact you there. Do you have another link or another way to chat?

Thanks for this awesome project.

Best.

I have captured dynamic images from the original video and the new video.
This is the original picture:

The following is the new picture:

Can you tell me which parameters need to add in order to make the generated video clearer?

Thank you very much!

For example, a question-answering system with a virtual human, after being converted through WAv2lip, produces almost real-time output

converting to final video
[vost#0:0 @ 0x86c61c0] Unknown encoder 'h264_nvenc'
[vost#0:0 @ 0x86c61c0] Error selecting an encoder
Error opening output file /home/Easy-Wav2Lip/temp/output.mp4.
Error opening output files: Encoder not found
Traceback (most recent call last):
File "/home/Easy-Wav2Lip/inference.py", line 694, in
main()
File "/home/Easy-Wav2Lip/inference.py", line 662, in main
subprocess.check_call([
File "/home/Easy-Wav2Lip/venv/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ffmpeg', '-y', '-loglevel', 'error', '-i', 'temp/result.mp4', '-i', '/home/Easy-Wav2Lip/temp/huyen-anh_01.wav', '-c:v', 'h264_nvenc', '/home/Easy-Wav2Lip/temp/output.mp4']' returned non-zero exit status 8.
Processing failed! :( see line above 👆
Processing failed

I run v8.2 with Colab to appear the following error.I use the video format is MP4, AVI, WMV, MPG. Video resolution is 480p, frame rate 24fps. Size 9:16

Traceback (most recent call last):
File "/content/Easy-Wav2Lip/run.py", line 98, in
in_width, in_height, in_fps, in_length = get_video_details(video_file)
File "/content/Easy-Wav2Lip/easy_functions.py", line 19, in get_video_details
info = json.loads(result.stdout)
File "/usr/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Hi! Help me, please. I'm trying to make a lip sync. I used to use your code on github, but now I'm getting an error about a second after step 2. I don't understand what's wrong. I decided to install Easy-wav2lip on my PC, I thought I would solve the error that way, but it persists. Can you please tell me what could be the problem?

i go to the aibl, download your all file can solve it . i also support u by charging :D

Hello

I have seen there is a version of inference that allow to have frame without face

Here a thread about it

Rudrabha/Wav2Lip#532

And some edited code
https://raw.githubusercontent.com/Rudrabha/Wav2Lip/master/inference.py

Could it be possible to integrate it into your project ?

Thanks

Have a good day

In create_tracked_mask and create_tracked_mask function code, mouth_detector is actually a face detector, however, the detector often fail to detect faces.