chathasphere / pno-ai Goto Github PK

This should go in the encoder module.

Vs. Position embedding.

Get additional metrics on loss

Wouldn't it be cool if you could play something on a MIDI controller and use it to seed the generator?

Is there a test metric to evaluate performance of the model?

Seems to struggle with the edge case of extra long notes.

Check out this line:

new_length = note.end - sample_start

seems problematic if new_length is greater than 30 seconds.

Hi. I was curious in trying your Music Transformer implementation, but I had to make some small changes to get it working.
Perhaps others may find some use from this information, and you might have insight.

First, in "train.py" line 147, there is a previously undeclared variable mask, which I believe should be x_mask.

Second, "generate.py" expects a "model.yaml" file that is not generated by "run.py": this was bypassed by directly setting the MusicTransformer model the same as in "run.py", and directly setting model_path to one of the checkpoints saved by "run.py"

Third, in "helpers.py", changed the line:
input_tensor = torch.LongTensor(input_sequence).unsqueeze(0)
into
model.cuda() input_tensor = torch.cuda.LongTensor(input_sequence).unsqueeze(0)
To fix an issue of mismatch between CPU and CUDA that arose in transformer.forward()

And lastly, because I was running this locally, I decreased batch size in run.py to fit my GPU.

Thanks.

...and figure out how to load them when generating

Successfully extracted note sequences w/ pedal action.
According to research , the next steps should be:

• Stretch music in time by: 0.95, 0.975, 1, 1.025, 1.05 (4x as many samples)
• Split MIDIs into 30-second clips
• Quantize samples (what exactly does this entail...sample rate of 125 Hz?
• Transpose all samples by all intervals up to a major third (9x as many samples)
• Simplify dynamics (midi velocity) to 32 bins

Woohoo, dev ops

Currently I dump a dictionary of parameters in the generate.py file. This could be more elegantly solved as a .yaml file.

Given the preprocessed data, convert these MIDI snippets to "Event Sequences."

Under the Time-shift representation, 413 events are possible:

• 128 note on events (each corresponding to a MIDI pitch)
• 128 note off events
• 125 time shift events (move time step forward by 8ms up to 1 second)
• 32 velocity events (change velocity applied to all subsequent events until another change)

A one-hot encoding turns each event into a 413-dimensional vector. A sequence of these is fed into a Neural Network.

MIT?

Adapt the Sequence Encoder class to generate Note sequences from poorly-behaved, generated event sequences. For instance, set "note off" automatically if no note off event is provided for a pitch.

How will this trade be affected by the various data augmentation strategies?

Docstrings for modules, plus an updated README/ Wiki. Links to a clean Colab notebook could be useful for teaching how to train a model.

• Use softmax sampling w/ temperature / topk to generate sequences
• Decode numeric sequence into event sequence
• Decode event sequence as MIDI object
• Write out MIDIs

Embed a trained model in a flask application, generate MIDI sequences on the fly

Hi, would it be possible to provide a link to a pre-trained model? Thanks!

Consider using numpy arrays instead of a list of notes?

Will likely lead to efficiency gains in transposing/stretching samples.

After running run.py and generate.py, it gave me the result that ‘no such file: saved_models/model.yaml’
I am doing research on the google magenta project (optimize the music transformer), your codes are valuable and useful for my research, so please help me to solve this problem.

preprocess.py should (probably) be converted to a class with an associated test module.

Have the ability to convert encoded event sequences back into pretty_midi Note sequences and write them out to a MIDI file.

This will be useful both for checking the fidelity of the encoding process as well as being able to listen to the model's eventual output!

There should not be an appreciable difference in training results with padded (sub-max length) sequences thrown in...this will require validating that the padding masks work as expected.

Useful to have a record of how training went, plus logs are more responsive to an instance shutting off

Some of the required steps will include:

-one-hot encoding the action sequences
-padding sequences to a uniform length for batching
-offsetting sequences for input/output pair ("hello worl", "ello world")

Think about how to initialize the training vector (?) All zeroes?
Helpful link(s):

• https://blog.floydhub.com/a-beginners-guide-on-recurrent-neural-networks-with-pytorch/
• https://towardsdatascience.com/writing-like-shakespeare-with-machine-learning-in-pytorch-d77f851d910c

play around with Colab!
is tpu an option? https://github.com/pytorch/xla/blob/master/contrib/colab/PyTorch_TPU_XRT_1_13.ipynb

Hi,

Is there a way to access the weights to obtain a pre-trained model? I can't train it on my machine but would still like to do fine-tuning.
The 'saved-model' folder seems to be empty...

Perhaps necessary if training on Colab. Save a model to gdrive every epoch.

Understand how it works!

some links:

• http://mlexplained.com/2019/02/15/building-an-lstm-from-scratch-in-pytorch-lstms-in-depth-part-1/
• https://github.com/keitakurita/Practical_NLP_in_PyTorch/blob/master/deep_dives/lstm_from_scratch.ipynb
• https://www.bioinf.jku.at/publications/older/2604.pdf
• https://blog.floydhub.com/long-short-term-memory-from-zero-to-hero-with-pytorch/