spectraldoy / musictransformertensorflow Goto Github PK

Hi,
First of all, I would like to thank you for such a well documented repository! It's a really valuable source to understand how Music Transformer works.

I'm trying to implement the Music_Transformer_Public.ipynb notebook on GPU. I have removed the TPU lines and I am using the train_step/val_step rather than distributed_train_step/distributed_val_step. But I couldn't understand some sections. I would be glad if you answer them:

• I couldn't figure out the if (batch + 1) % 202 == 0 or batch == 0: line in the training section. Why the number 202? Should I change it according to my dataset?
• Which one is more common while talking about the training a transformer: epoch number or total training steps? Currently while training my model I'm setting the epoch number as a quite high number and I'm looking at the total training steps and training loss.
• Although you have created a test dataset, you haven't used it in the notebook if I'm not mistaken. Have you implemented the evaluation on somewhere else? Or should I ask, is it necessary to test the trained transformer model?
• And you have written val_step() function but didn't use it either. I'm guessing that, it's usage is just like train_step() function. So I'm feeding the validation dataset to val_step() in the training loop section. Am I doing it right?
• Lastly, I'd like to ask you how many hours it took you to train 100.000 training steps on TPU?

Thanks in advance!
Fansesi