ricardokleinklein / deepmultispeech Goto Github PK

FUNDAMENTAL

• Repair data: download clean, full dataset and merge data already available.
  It's been observed that there are barely any sentence common to all the speakers, contrary to what the website for noisy-VCTK corpus says. Therefore, for parallel VC it is important to have all the data. Otherwise the number of samples available will be ridiculous.
• Make Noisy-VCTK automatic pipeline for downloading data.
• New metadata processing - following original tree structure.

Input data:

• SE training on 4 target speakers
• SE test on 2 test speakers
• VC training on 2? source speakers and same 4 target speakers
• VC test on 2 test speakers and same 4 target speakers
• VC as parallel-VC (same utterance source-target)
• DTW between source Mel and target Mel
• TTS training on same 4 target speakers
• TTS test on 2 test speakers
• User-defined number of source speakers in VC
• User-defined number of target speakers in SE/VC

Architecture:

• SE modality
• VC modality
• TTS modality
• Body with several projection layers
• Include non-linearities Body
• Change dimensionality at different layers - Change of concept

Training:

• Split training of modalities - rest of model (speedup?)
• Train jointly
• Extract feature maps obtained after modality nets
• Extract feature maps obtained after body

Eval:

• Run SNR on SE results