A custom cost function may be useful in rewarding and punishing the network. For example, positive behaviours to reinforce may be repeating phrases in different bars ("motifs") while a negative behaviour would be to repeat a single note for too long.

Since RNNs have difficulties dealing with long input sequences, consider experimenting with an attention mechanism. This may improve results since songs usually consist of phrases over several beats, and encoding the data across multiple bars may lead to loss of information.