Comments (13)
Hello, thank you for the clear and detailed description of the issue. I think I clearly understand the issue. It is caused by this line:
Line 97 in 0600d0f
This line smooths bounding boxes over a 5 frame window. This is desirable as face detection can be at times be noisy and change a lot across frames. But, when there is rapid motion, the face position changes so much that the averaging the box location across the frames is undesirable.
We will look into this issue and update you once there is some sort of fix.
from wav2lip.
Please consider to mention the use of
--nosmooththis
on the main GitHub in case people would like to use it.
Will do it right away!
from wav2lip.
Can you pull the latest commit, and try with the --nosmooth
argument and let me know?
from wav2lip.
Just to be clear: I'm a newbie when it comes to GitHub and I'm not a programmer so everything is very new to me, all I can do is to follow instructions and tutorials.
So I hope that I did it correct:
I've downloaded "inference.py" made a backup of the old one, and put this on the main master directory.
Then I used the same files I did for the test of this issue and at the end I've added --nosmooth
like this:
python inference.py --checkpoint_path checkpoints/wav2lip_gan.pth --face MyDir/video.avi --audio MyDir/audio.wav --outfile MyDir/result_video.mp4 --nosmooth
but I get this error:
inference.py: error: unrecognized arguments: --nosmooth
I double checked that I put the NEWEST file (from about 30 minutes ago) instead of the old one, but still it seems like it's not recognizing the new command.
Maybe I did something wrong, not sure if I'm the right man to help testing but I can try if it's something simple enough for me to follow, sorry for the mess.
from wav2lip.
Hey, I had a similar issue. I ran it with the --nosmooth
argument and it ran till a 100% but then gave me the following error:
Traceback (most recent call last):
File "inference.py", line 280, in <module>
main()
File "inference.py", line 250, in main
total=int(np.ceil(float(len(mel_chunks))/batch_size)))):
File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1127, in __iter__
for obj in iterable:
File "inference.py", line 111, in datagen
face_det_results = face_detect(frames) # BGR2RGB for CNN face detection
File "inference.py", line 101, in face_detect
results = [[image[y1: y2, x1:x2], (y1, y2, x1, x2)] for image, (x1, y1, x2, y2) in zip(images, boxes)]
UnboundLocalError: local variable 'boxes' referenced before assignment
I see that line 101 uses boxes
Line 101 in 8549b79
which we only retrieve if line 100 is true otherwise we don't set it to anything.
Line 100 in 8549b79
from wav2lip.
@AlonDan can you check now if it is better?
from wav2lip.
Sure, I just tried it now it won't start running there is an error right after I press ENTER:
File "inference.py", line 6
<!DOCTYPE html>
^
SyntaxError: invalid syntax
Even if I just run the main command:
python inference.py
I get the same error.
from wav2lip.
Hello, I think this is an error from your end, as you can see that the repo does not have that in line 6:
Line 6 in 11a8eac
from wav2lip.
Hmm, if that's the case I'm not sure how to fix that as I explained unfortunatly I'm a newbie in this.
f I switch back to the backup (old file version before the issue) everything works, but of course without the --nosmooth
Any suggestions to fix that issue on my side so I can test the new file version?
Thanks ahead.
from wav2lip.
Any suggestions to fix that issue on my side so I can test the new file version?
download the latest version once more. or the whole repo once more.
from wav2lip.
GOOD NEWS!
I just downloaded the latest file version again and did a test again, from the SAME video as the original issue, now it's STEADY as it should be, probably need to test on more complicated video situations but from the current one it's IMPRESSIVE!
Great job, thank you for the super quick fix!
here is the current version in case anyone would like to see the difference with --nosmooth
The mouth now following every single frame!
DOWNLOAD same video using --nosmooth
as (zip file)
result_video (nosmooth enabled).zip
Suggestion:
Please consider to mention the use of --nosmooththis
on the main GitHub in case people would like to use it.
Same Frame with --nosmooth and without:
from wav2lip.
I'm seeing weird artifacts near the lips for some videos even after using --nosmooth
. Is this because I am using high res videos?
from wav2lip.
It’s hard to say without seeing the video. But, yes, high res videos and faces can produce artefacts.
from wav2lip.
Related Issues (20)
- 请问一下训练完这个项目的评价指标怎么查看呢
- wav2lip288x288 HOT 2
- 训练syncnet报错:RuntimeError: DataLoader worker (pid 899437) is killed by signal: Segmentation fault. ,训练到第290000结束后报错:,请问有人遇到过这个问题吗? HOT 1
- FileNotFoundError: [WinError 2] the specified file cannot be found HOT 2
- IndexError: list index out of range HOT 1
- Strange line in chin
- 有出训练好的模型的吗,联系我
- Could not build wheels for opencv-contrib-python HOT 1
- 按照官方步骤从0开始训练CMLR中文数据集,结果最终视频和音频对应不上 HOT 2
- HELPPP ERROR ModuleNotFoundError: No module named 'ffmpeg' HOT 4
- Who needs high-quality lip sync - contact me! HOT 2
- 我的研究进展
- Higher Temporal Context window for Syncnet
- wav2lip not working for me please help!!! HOT 2
- First step not working for me! HOT 2
- Question abount get_segmented_mels
- Traceback (most recent call last): File "inference.py", line 280, in <module> main() File "inference.py", line 250, in main total=int(np.ceil(float(len(mel_chunks))/batch_size)))): File "D:\lq\anaconda3\envs\wav2lip\lib\site-packages\tqdm\std.py", line 1127, in __iter__ for obj in iterable: File "inference.py", line 113, in datagen face_det_results = face_detect(frames) # BGR2RGB for CNN face detection File "inference.py", line 92, in face_detect raise ValueError('Face not detected! Ensure the video contains a face in all the frames.') ValueError: Face not detected! Ensure the video contains a face in all the frames.
- 出wav2lip高清通用模型256尺寸,包含推理代码,oonx加速推理。需要微信 tke365
- wav2lip [mp3 @ 000002a6d5b45d80] Estimating duration from bitrate, this may be inaccurate HOT 2
- 出高清模型384尺寸泛化 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wav2lip.