Pipeline to take one video and one subtitle file to generate a new audio/video file taking the video stream of the first one and the generated audio/speech from subtitle file using Coqui-ai TTS and synchronize the audio timing according to subtitle time.

As post-processing, any extra silent parts of the audio/video generated are shrinked.

ffmpeg, pydub, librosa, coqui-ai TTS, ffmpeg-python, sub-to-audio

ffmpeg on linux

apt-get install ffmpeg

virtual environment

python3.9 -m venv .

. bin/activate
python3 pipeline.py video/lecture-1.1.mp4 srt/lecture-1.1-captions.mod.srt