• Lecture and seminar materials for each week are in ./week* folders, see README.md for materials and instructions
• Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue
• The current version of the course is conducted in autumn 2022 at the CS Faculty of HSE

• week01 Introduction to Course

  • Lecture: Introduction to Course
  • Seminar: Intro in pytorch

• week02 Introduction to Digital Signal Processing

  • Lecture: Signals, Fourier Transform, spectrograms, MelScale, MFCC
  • Seminar: DSP in practice, spectrogram creation, training a model for audio MNIST

• week03 Speech Recognition I

  • Lecture: Metrics, datasets, Connectionist Temporal Classification (CTC), Listen Attend and Spell (LAS), Beam Search
  • Seminar: Audio Augmentations, Beam Search

• week04 Speech Recognition II

  • Lecture: RNN-T, language model fusion, Byte-Pair Encoding (BPE)
  • Seminar: Forced Alignment

• week05 Speaker verification and identification

  • Lecture: Metric Learning: Cosine, Contrastive, Triplet Losses. Angular Softmax. ArcFace
  • Seminar: Generalized End2End Loss for Speaker Verification

• week06 Key-word spottind

  • Lecture: (DNN, CNN, RNN+Attention) based KWS, SVDF, Orthogonality Regularization and other Tricks
  • Seminar: CNN+Attention+RNN KWS model

• week07 Text to Speech (TTS)

  • Lecture: Tacotron, DeepVoice, GST, FastSpeech, AdaSpeech, Attention Tricks
  • Seminar: FastSpeech I

• week08 Neural Vocoders

  • Lecture: WaveNet, Parallel WaveGAN
  • Seminar: WaveNet

• week09 Advanced TTS and Vocoders

  • Lecture: Introduction into generative models. ParallelWaveNet, WaveGlow, WaveFlow, MelGAN, HiFiGAN

• week10 Voice Conversion

  • Lecture: Disentanglement & Direct based methods,
  • Seminar: Homework Q&A

• week11 Self-supervision in Audio and Speech

• week12 Invited talks

• ASR Training speech recognition model
• KWS Implementation of KWS model
• TTS Implementation of TTS model
• NV Implementation of Neural Vocoder Model

• Lecture recordings on YouTube (in russian): YouTube

Course materials and teaching performed by

• Alexander Markovich
• Daniil Ivanov
• Timofey Smirnov
• Alexander Mamaev