This repository comprises a compilation of the objective metrics used in several text-to-speech (TTS) papers.

First, clone and enter the repo:

git clone https://github.com/AI-Unicamp/TTS-Objective-Metrics
cd TTS-Objective-Metrics

Install dependencies:

pip install -r requirements.txt

Then, configure the global parameters as you wish in config-> global.py. If not done, all statistics will be computed with the default parameters already set.

The main usage of the repo is to calculate all available metrics for a batch of (ground truth, synthesized) audio pairs (test/eval set). For this make sure you have the (ground_truth, synthesized) audio pairs names matching and in numbered order, each with the same number of digits of the greatest file (eg. if there are 100 files, you shall start with 000.wav, if there are [10,99] you shall start with 00.wav). as in:

📂My Audio
┣ 📂Ground Truths
┃ ┣ 📜00.wav
┃ ┣ 📜01.wav
┃ ┣ ...
┣ 📂Synthesizeds
┃ ┣ 📜00.wav
┃ ┣ 📜01.wav
┃ ┣ ...

Then, choose one pitch computing algorithm and run the following command:

  python -m bin.compute_metrics --gt_folder_path 'My Audio\Ground Truths' --synth_folder_path 'My Audio\Synthesizeds' --pitch_algorithm 'yin'

The result will be saved in a file named metrics.json in the main repo folder: 'TTS Objective Metrics/metrics.json'.

Alternatively, it is possible to calculate a single metric for a pair of (ground truth, synthesized) audio, by choosing an available metric and pitch computation method (yin or dio) with one of the following commands:

# For DTW, FFE, GPE, VDE, moments
python -m metrics.DTW --gt_path 'ground_truth_audio.wav' --synth_path 'synthesized_audio.wav' --pitch_algorithm 'yin'

# For MSD or MCD   
python -m metrics.MSD --gt_path 'path_to_ground_truth_audio.wav' --synth_path 'path_to_synthesized_audio.wav'           

The result will be displayed in the terminal.

📦TTS Objective Metrics
┣ 📂audio
┃ ┣ 📜helpers.py
┃ ┣ 📜pitch.py
┃ ┣ 📜visuals.py
┣ 📂bin
┃ ┣ 📜compute_metrics.py
┣ 📂config
┃ ┣ 📜global_config.py
┣ 📂metrics
┃ ┣ 📜dists.py
┃ ┣ 📜DTW.py
┃ ┣ 📜FFE.py
┃ ┣ 📜GPE.py
┃ ┣ 📜helpers.py
┃ ┣ 📜MCD.py
┃ ┣ 📜moments.py
┃ ┣ 📜MSD.py
┃ ┣ 📜VDE.py
┣ 📜README.md

As the repo is still in its infancy, feel free to either open an issue, discussion or send a pull request, or even contact us by e-mail.

• Leonardo B. de M. M. Marques ([email protected])
• Lucas Hideki Ueda ([email protected])

• Coqui-AI
• Facebook Fairseq
• NVIDIA Deep Learning Examples
• NVIDIA Mellotron
• MAPS
• Yin

All references are listened on top of the used code itself.