Pipeline to take one video and one subtitle file to generate a new audio/video file taking the video stream of the first one and the generated audio/speech from subtitle file using Coqui-ai TTS and synchronize the audio timing according to subtitle time.
As post-processing, any extra silent parts of the audio/video generated are shrinked.
ffmpeg, pydub, librosa, coqui-ai TTS, ffmpeg-python, sub-to-audio
ffmpeg on linux
apt-get install ffmpeg
virtual environment
python3.9 -m venv .
. bin/activate
python3 pipeline.py video/lecture-1.1.mp4 srt/lecture-1.1-captions.mod.srt