Comments (2)
Hi @roudimit ,
Thank you so much for raising this issue and so sorry for the late reply!
To be honest, I never tested our checkpoints on VSR since it was out-of-scope! However, looking at the video processing code for muavic and av-hubert, I can see there are a few differences:
- how frames are extracted from the video, av-huberts does this on the fly. MuAViC does it beforehand.
- how video is saved, both uses
ffmpeg
but a bit differently.
These are the only differences that I could find! Hope this helps.
from muavic.
Thanks @Anwarvic for the pointers! I tested the video loading and the video saving. The loading functions from MuAViC and AV-HuBERT load the video the same. However, the saving using ffmpeg
is different since AV-HuBERT specifies '-crf', '20'
, while MuAViC saving uses the default (I belief crf=23), which means the video frames from MuAViC are more compressed. A link for more details: https://stackoverflow.com/questions/64011346/ffmpeg-quality-conversion-options-video-compression
I'm going to leave this issue open so that others are aware of the difference between the video processing.
from muavic.
Related Issues (20)
- Minor issue HOT 2
- Error when preprocessing the video data HOT 1
- A small bug during audio pre-processing HOT 1
- Got error when preparing LRS3 HOT 5
- download_ted2020() error HOT 4
- TEDx Talk with ID=D4TE28-L7FI is not available anymore HOT 5
- Error running the data prep script HOT 7
- Error when generating the manifest for AVSR HOT 3
- Questions towards hyper-parameters and the token post-processing HOT 1
- Unable to download corpora other than English HOT 1
- Problems when Downloading the Italian Dataset HOT 2
- Empty X -> EN translations HOT 2
- Noise parameters for decoding and training HOT 6
- Multilingual AVSR model decoding and training HOT 2
- Problem met when downloading German data HOT 2
- Only audio files could be downloaded
- Could you please tell me what version your 'sox' is? HOT 3
- How much storage do I need in total to download the muavic dataset?
- RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for AVHubertSeq2Seq: HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from muavic.