01 DDSP: Differentiable Digital Signal Processing

• 00:00:00 Introduction
• 00:00:30 Magenta Research Project
• 00:01:14 DDSP Library Overview
• 00:01:36 DDSP Modules
• 00:02:29 DDSP Processors Module
• 00:04:16 DDSP Core Module
• 00:06:41 DDSP Spectral Ops Module
• 00:07:46 DDSP Original Paper

02 Differentiable Digital Signal Processing: Timbre Transfer - Deep Learning

• 00:00:00 Timbre Transfer Demo
• 00:02:55 Installing and Importing the DDSP Library
• 00:03:10 Recording or Uploading Audio
• 00:05:08 Loading a Model: Violin
• 00:06:56 Resynthesizing Audio
• 00:08:01 Modifying Conditioning
• 00:09:06 Pre-trained Trumpet Model
• 00:10:22 Pre-trained Flute2 Model
• 00:11:18 Pre-trained Tenor Saxophone Model
• 00:12:56 Pitch Shifting
• 00:13:50 Demo Application Conclusions
• 00:15:19 DDSP ICLR Paper: The International Conference on Learning Representations
• 00:17:18 Install and Imports
• 00:18:16 Gin Config Framework
• 00:22:35 Recording Audio
• 00:26:23 base64: Decode the Base64 encoded bytes-like object or ASCII string
• 00:26:43 Audio bytes to Numpy
• 00:27:44 Uploading an Audio File
• 00:29:29 Colab Files Upload
• 00:33:43 Log Magnitude Spectrogram
• 00:36:40 DDSP Spectral Operations: Compute Magnitude
• 00:37:11 DDSP Spectral Operations: Short-time Fourier Transform (STFT)
• 00:38:15 Tensorflow Signal: Short-time Fourier Transform (STFT)
• 00:39:37 DDSP Safe Log
• 00:43:14 Numpy Rotate 90
• 00:44:56 Pyplot Matshow
• 00:46:33 DDSP Colab Utils: Play Audio
• 00:49:52 Scipy IO Wavfile Write
• 00:51:37 IPython Display Audio
• 00:55:29 Computing Audio Features
• 00:56:28 CREPE: A Convolutional Representation for Pitch Estimation
• 00:57:39 CREPE ICASSP 2018 Paper
• 01:06:16 DDSP Timbre Transfer Audio Features
• 01:09:28 Computing F0
• 01:10:23 Viterbi Algorithm
• 01:13:38 Computing Loudness
• 01:17:50 Dynamics, Intensity and Loudness
• 01:21:12 A-weighting Function
• 01:24:59 Plotting the Audio Features
• 01:25:28 Loading a Pre-Trained Model
• 01:27:05 Gsutil Tool
• 01:28:22 tf.io.gfile
• 01:29:19 Parsing Gin Config
• 01:31:03 Gin Query Parameters
• 01:35:57 Understanding the Pre-Trained Model
• 01:36:16 DDSP Autoencoder Model
• 01:37:13 DDSP Model Base Class
• 01:39:01 DDSP ICLR Paper: Autoencoder Architecture
• 01:39:43 Gin Config - Model Parameters
• 01:42:17 DDSP Encoders
• 01:44:12 DDSP Decoders
• 01:45:47 Recurrent Neural Network (RNN)
• 01:46:18 Gated Recurrent Units (GRUs)
• 01:47:28 Harmonic Oscillator / Additive Synthesizer
• 01:48:54 Filtered Noise / Subtractive Synthesizer
• 01:50:08 Multi-Scale Spectral Loss
• 01:51:52 DDSP Reverb
• 01:52:21 Conclusion

03 Tensorflow + Tensorboard + Scikit-learn: Simple audio recognition: Recognizing keywords

• 00:00:00 Introduction
• 00:00:34 Simple audio recognition: Recognizing keywords Example
• 00:03:53 Tensorflow Keras
• 00:05:38 Tensorflow
• 00:07:44 Seed for Experiment Reproducibility
• 00:09:56 Speech Commands Dataset
• 00:11:33 Downloading files using 'tf.keras.utils.get_file'
• 00:15:07 Returning a list of files that match the given pattern(s) using 'tf.io.gfile.glob'
• 00:16:19 Splitting the dataset into training, validation and testing subsets
• 00:19:29 Scikit-learn
• 00:20:14 Stratified ShuffleSplit
• 00:24:38 Writing descriptive and efficient input pipelines using 'tf.data.Dataset'
• 00:26:20 Reading the contents of a file using 'tf.io.read_file'
• 00:27:14 Applying transformations across the elements of a dataset using '.map(...)'
• 00:31:18 Spectrograms using 'tf.signal.stft'
• 00:31:32 Zero-padding in Tensorflow
• 00:34:00 Listening to audio files
• 00:36:10 Plotting Spectrograms
• 00:40:29 Batching the training and validation subsets using '.batch(...)'
• 00:41:48 Caching elements of a dataset using '.cache(...)'
• 00:42:20 Prefetching elements of a dataset unsing 'prefetch(...)'
• 00:43:13 Preprocessing Layers: Resizing and Normalization
• 00:43:48 Building a Sequential Model in Tensorflow Keras
• 00:46:10 Printing a string summary of the network using '.summary()'
• 00:46:46 Configuring the model for training using '.compile(...)'
• 00:48:12 Configuring Tensorboard
• 00:50:18 Training the Model using '.fit(...)'
• 00:50:33 Visualization in Tensorboard
• 00:52:20 Evaluating the Test set Performance
• 00:53:39 Displaying a Confusion Matrix using '.tf.math.confusion_matrix(...)'
• 00:54:40 Running Inference on an Audio File