liuhaumin / leadsheetarrangement Goto Github PK

Lead Sheet Generation and Arrangement 自動簡譜編曲

Home Page: https://liuhaumin.github.io/LeadsheetArrangement/

Python 3.44% Jupyter Notebook 96.56%

leadsheetarrangement's Introduction

LeadsheetArrangement 自動簡譜編曲 🎵

Lead Sheet Arrangement is a task to automatically accompany the existed or generated lead sheets with multiple instruments. The rhythm, pitch notes, onsets and even playing style of different instruments are all learned by the model.

We train the model with Lakh Pianoroll Dataset (LPD) to generate pop song style arrangement consisting of bass, guitar, piano, strings and drums.

Sample results are available here.

Papers

Lead sheet generation and arrangement by conditional generative adversarial network
Hao-Min Liu and Yi-Hsuan Yang, to appear in International Conference on Machine Learning and Applications (ICMLA), 2018. [arxiv]

Lead Sheet Generation and Arrangement via a Hybrid Generative Model
Hao-Min Liu*, Meng-Hsuan Wu*, and Yi-Hsuan Yang (*equal contribution)
in ISMIR Late-Breaking Demos Session, 2018. (non-refereed two-page extended abstract)
[paper] [poster]

Lead sheet and Multi-track Piano-roll generation using MuseGAN
Hao-Min Liu, Hao-Wen Dong, Wen-Yi Hsiao and Yi-Hsuan Yang, in GPU Technology Conference (GTC), 2018. [poster]

Usage

Approach 1: Run pretrained model using Scripts

Go to folder (./data/preprocessing/)
Put your file in folder (./data/preprocessing/mysong_mid_C/) with name specified as "test.mid"
- Note that test.mid should be in C key and two tracks(melody and chord)
Back to root directory and run scripts.py
Find the outputs in folder (./output/)

Approach 2: Train or test the model step by step by your own

Step 0: preprocessing the midi data

Go to folder (./data/preprocessing/)
Put your input song_name.midi file in folder (./data/preprocessing/mysong_mid_C/)
- Note that song_name.midi should be in C key and two tracks(melody and chord)
Back to folder (./preprocessing/)
Run pianoroll_mysong.ipynb
- Remember to change filenames in code lines into "song_name"
- Output files will be stored in folder(./data/chord_roll/val/)with name as "x_bar_chroma_song_name.npy" and "y_bar_chroma_song_name.npy"

Step 1: Loading the data

Open file store_sa.py
Turn filename in code line # 36 into your filename, i.e. ('y_bar_chroma_song_name.npy')
Load the data by running store_sa.py

Step 2: adjust training or testing modes in main.py

import tensorflow as tf
from musegan.core import MuseGAN
from musegan.components import NowbarHybrid
from config import *

# Initialize a tensorflow session

""" Create TensorFlow Session """
with tf.Session() as sess:
    
    # === Prerequisites ===
    # Step 1 - Initialize the training configuration        
    t_config = TrainingConfig
    t_config.exp_name = 'exps/nowbar_hybrid'        

    # Step 2 - Select the desired model
    model = NowbarHybrid(NowBarHybridConfig)
    
    # Step 3 - Initialize the input data object
    input_data = InputDataNowBarHybrid(model)
    
    # Step 4 - Load training data
    path_x_train_bar = 'tra_X_bars'
    path_y_train_bar = 'tra_y_bars'
    input_data.add_data_sa(path_x_train_bar, path_y_train_bar, 'train') # x: input, y: conditional feature
    
    # Step 5 - Initialize a museGAN object
    musegan = MuseGAN(sess, t_config, model)
    
    # === Training ===
    musegan.train(input_data)

    # === Load a Pretrained Model ===
    musegan.load(musegan.dir_ckpt)

    # === Generate Samples ===
    path_x_test_bar = 'val_X_bars'
    path_y_test_bar = 'val_y_bars'
    input_data.add_data_sa(path_x_test_bar, path_y_test_bar, key='test')
    musegan.gen_test(input_data, is_eval=True)

Step 3: run main.py

Training mode
- Checkpoints are stored in folder (./exps/nowbar_hybrid/checkpoint/)
Testing mode
- the output files are stored in folder (./exps/nowbar_hybrid/gen/)

Step 4: postprocessing the output midi

Go to folder (./postprocessing/)
Run file npy2mid.ipynb
- Remember to change filenames in code lines into "song_name"
- Output files are stored in folder (./output/)

leadsheetarrangement's People

Contributors

Stargazers

Watchers

Forkers

jingguo127 aozhi hongzetian wang1ang fengyizhou artemuzz amit2014 markhsia liuchengblog hanhaodizhang worse2worst nayeunk thzdyjy kimleekyung hchen605

leadsheetarrangement's Issues

ValueError: Cannot feed value of shape (64, 48, 84, 2) for Tensor 'NowbarHybrid/x:0', which has shape '(?, 48, 84, 5)'

Error while installing

When installing:

Step: Seeding basic data for LeadBrowser CRM kickstart...
Seeding: LeadBrowser\Admin\Database\Seeders\DatabaseSeeder
Seeding: LeadBrowser\Admin\Database\Seeders\AttributeSeeder

In Connection.php line 712:
                                                                                                               
  SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count   
  at row 8 (SQL: insert into `attributes` (`class`, `code`, `created_at`, `entity_type`, `id`, `info`, `is_re  
  quired`, `is_unique`, `is_user_defined`, `is_visible`, `lookup_type`, `name`, `quick_add`, `sort_order`, `t  
  ype`, `updated_at`, `validation`) values (col-sm-12 col-md-6, title, 2022-12-30 09:44:49, leads, 1, ?, 1, 0  
  , 0, 1, ?, Title, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, description, 2022-12-30 09:44:4  
  9, leads, 2, ?, 0, 0, 0, 1, ?, Description, 1, 2, textarea, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, l  
  ead_value, 2022-12-30 09:44:49, leads, 3, ?, 1, 0, 0, 1, ?, Lead Value, 1, 3, price, 2022-12-30 09:44:49, ?  
  ), (col-sm-12 col-md-6, lead_source_id, 2022-12-30 09:44:49, leads, 4, ?, 1, 0, 0, 1, lead_sources, Source,  
   1, 4, select, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, lead_type_id, 2022-12-30 09:44:49, leads, 5, ?  
  , 1, 0, 0, 1, lead_types, Type, 1, 5, select, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, user_id, 2022-1  
  2-30 09:44:49, leads, col-sm-12 col-md-6, ?, 0, 0, 0, 1, users, Sales Owner, 1, 7, select, 2022-12-30 09:44  
  :49, ?), (col-sm-12 col-md-6, expected_close_date, 2022-12-30 09:44:49, leads, 7, ?, 0, 0, 0, 1, ?, Expecte  
  d Close Date, 1, 8, date, 2022-12-30 09:44:49, ?), (lead_pipeline_stage_id, 2022-12-30 09:44:49, leads, 8,   
  0, 0, 0, lead_pipeline_stages, Stage, 0, 9, select, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, name, 202  
  2-12-30 09:44:49, persons, 9, ?, 1, 0, 0, 1, ?, Name, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-m  
  d-6, emails, 2022-12-30 09:44:49, persons, 10, ?, 1, 1, 0, 1, ?, Emails, 1, 2, email, 2022-12-30 09:44:49,   
  ?), (col-sm-12 col-md-6, contact_numbers, 2022-12-30 09:44:49, persons, 11, ?, 0, 1, 0, 1, ?, Contact Numbe  
  rs, 1, 3, phone, 2022-12-30 09:44:49, numeric), (col-sm-12 col-md-6, organization_id, 2022-12-30 09:44:49,   
  persons, col-sm-12 col-md-12, ?, 0, 0, 0, 1, organizations, Organization, 1, 4, lookup, 2022-12-30 09:44:49  
  , ?), (col-sm-12 col-md-6, name, 2022-12-30 09:44:49, organizations, 13, ?, 1, 1, 0, 1, ?, Name, 1, 1, text  
  , 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, address, 2022-12-30 09:44:49, organizations, 14, ?, 0, 0, 0  
  , 1, ?, Address, 1, 2, address, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, name, 2022-12-30 09:44:49, pr  
  oducts, 15, ?, 1, 0, 0, 1, ?, Name, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, description,   
  2022-12-30 09:44:49, products, 16, ?, 0, 0, 0, 1, ?, Description, 1, 2, textarea, 2022-12-30 09:44:49, ?),   
  (col-sm-12 col-md-6, sku, 2022-12-30 09:44:49, products, 17, ?, 1, 1, 0, 1, ?, SKU, 1, 3, text, 2022-12-30   
  09:44:49, ?), (col-sm-12 col-md-6, quantity, 2022-12-30 09:44:49, products, 18, ?, 1, 0, 0, 1, ?, Quantity,  
   1, 4, number, 2022-12-30 09:44:49, numeric), (col-sm-12 col-md-6, price, 2022-12-30 09:44:49, products, 19  
  , ?, 1, 0, 0, 1, ?, Price, 1, 5, text, 2022-12-30 09:44:49, decimal), (col-sm-12 col-md-6, user_id, 2022-12  
  -30 09:44:49, quotes, 20, ?, 1, 0, 0, 1, users, Sales Owner, 1, 1, select, 2022-12-30 09:44:49, ?), (col-sm  
  -12 col-md-6, subject, 2022-12-30 09:44:49, quotes, 21, ?, 1, 0, 0, 1, ?, Subject, 1, 2, text, 2022-12-30 0  
  9:44:49, ?), (col-sm-12 col-md-6, description, 2022-12-30 09:44:49, quotes, 22, ?, 0, 0, 0, 1, ?, Descripti  
  on, 1, 3, textarea, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, billing_address, 2022-12-30 09:44:49, quo  
  tes, 23, ?, 1, 0, 0, 1, ?, Billing Address, 1, 4, address, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, sh  
  ipping_address, 2022-12-30 09:44:49, quotes, 24, ?, 0, 0, 0, 1, ?, Shipping Address, 1, 5, address, 2022-12  
  -30 09:44:49, ?), (col-sm-12 col-md-6, discount_percent, 2022-12-30 09:44:49, quotes, 25, ?, 0, 0, 0, 1, ?,  
   Discount Percent, 1, col-sm-12 col-md-6, text, 2022-12-30 09:44:49, decimal), (col-sm-12 col-md-6, discoun  
  t_amount, 2022-12-30 09:44:49, quotes, 26, ?, 0, 0, 0, 1, ?, Discount Amount, 1, 7, price, 2022-12-30 09:44  
  :49, decimal), (col-sm-12 col-md-6, tax_amount, 2022-12-30 09:44:49, quotes, 27, ?, 0, 0, 0, 1, ?, Tax Amou  
  nt, 1, 8, price, 2022-12-30 09:44:49, decimal), (col-sm-12 col-md-6, adjustment_amount, 2022-12-30 09:44:49  
  , quotes, 28, ?, 0, 0, 0, 1, ?, Adjustment Amount, 1, 9, price, 2022-12-30 09:44:49, decimal), (col-sm-12 c  
  ol-md-6, sub_total, 2022-12-30 09:44:49, quotes, 29, ?, 1, 0, 0, 1, ?, Sub Total, 1, 10, price, 2022-12-30   
  09:44:49, decimal), (col-sm-12 col-md-6, grand_total, 2022-12-30 09:44:49, quotes, 30, ?, 1, 0, 0, 1, ?, Gr  
  and Total, 1, 11, price, 2022-12-30 09:44:49, decimal), (col-sm-12 col-md-6, expired_at, 2022-12-30 09:44:4  
  9, quotes, 31, ?, 1, 0, 0, 1, ?, Expired At, 1, col-sm-12 col-md-12, date, 2022-12-30 09:44:49, ?), (col-sm  
  -12 col-md-6, person_id, 2022-12-30 09:44:49, quotes, 32, ?, 1, 0, 0, 1, persons, Person, 1, 13, lookup, 20  
  22-12-30 09:44:49, ?), (col-sm-12 col-md-6, title, 2022-12-30 09:44:49, search_locations, 33, ?, 1, 0, 0, 1  
  , ?, Key, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-3, type_id, 2022-12-30 09:44:49, search_lo  
  cations, 34, ?, 0, 0, 0, 2, search_types, Industry, 1, 5, select, 2022-12-30 09:44:49, ?), (col-sm-6 col-md  
  -6, expected_items, 2022-12-30 09:44:49, search_locations, 35, The maximum number of results you want to ge  
  t. Remember that each result will be charged to your wallet., 1, 0, 0, 1, ?, Quantity, 1, 4, text, 2022-12-  
  30 09:44:49, numeric), (col-sm-6 col-md-3, phrase, 2022-12-30 09:44:49, search_locations, 36, Decimal words  
   that should appear next to the result or on the website of the business you want to find. For example, if   
  you are looking for a company dealing with new technologies, you could enter, for example, Rust or GO., 0,   
  0, 0, 2, ?, Phrase, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-12, location, 2022-12-30 09:44:4  
  9, search_locations, 37, ?, 1, 0, 0, 1, ?, Location, 1, 2, location, 2022-12-30 09:44:49, ?), (col-sm-12 co  
  l-md-6, population_id, 2022-12-30 09:44:49, search_locations, 38, ?, 0, 0, 0, 2, population_sources, Popula  
  tion, 1, 4, select, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, duplicates, 2022-12-30 09:44:49, search_l  
  ocations, 39, Duplicates give you an insight into the headquarters of companies. For example, with this opt  
  ion you can download data of all premises of company X in the city of Y., , 0, 0, 2, ?, Duplicates, 1, 4, c  
  heckbox, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, urls, 2022-12-30 09:44:49, search_websites, 40, Add   
  page addresses so that each is with a decimal point, 0, 0, 0, 1, ?, Urls, 1, 1, text, 2022-12-30 09:44:49,   
  ?), (col-sm-12 col-md-6, phones, 2022-12-30 09:44:49, search_phones, 41, Add phones addresses so that each   
  is with a decimal point, 0, 0, 0, 1, ?, Phones, 1, 1, text, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, e  
  commerce, 2022-12-30 09:44:49, search_locations, 42, Only show results that are e-commerce, , 0, 0, 2, ?, E  
  -commerce, 1, 4, checkbox, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, emails, 2022-12-30 09:44:49, searc  
  h_locations, 43, , , 0, 0, 2, ?, Emails, 1, 4, checkbox, 2022-12-30 09:44:49, ?), (col-sm-12 col-md-6, work  
  ers, 2022-12-30 09:44:49, search_locations, 44, , , 0, 0, 2, ?, Workers, 1, 4, checkbox, 2022-12-30 09:44:4  
  9, ?))

Couldn't run your pretrained model using Scripts

--------------Start arrangement generation !!--------------
*running sampler...
Traceback (most recent call last):
File "scripts.py", line 4, in
import main
File "/home/ohla-ai/Projects/LeadsheetArrangement/main.py", line 62, in
musegan.gen_test(input_data, is_eval=True)
File "/home/ohla-ai/Projects/LeadsheetArrangement/model/core.py", line 447, in gen_test
type_=type_)
File "/home/ohla-ai/Projects/LeadsheetArrangement/model/core.py", line 412, in run_sampler
self.model.prediction_chroma], feed_dict=feed_dict)
File "/home/ohla-ai/anaconda3/envs/tf15/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/home/ohla-ai/anaconda3/envs/tf15/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1156, in _run
(np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (64, 48, 84, 2) for Tensor 'NowbarHybrid/x:0', which has shape '(?, 48, 84, 5)'

Above is what I am occured, my input is a 2 track 8 bar midi file with melody and chords. Need your help please...

code refactor and library modules version update

@liuhaumin
How to experiment this project now?

conda create env failed.

midi2audio==0.1.1
imageio==2.1.2
mido==1.2.8
pretty-midi==0.2.8
pypianoroll==0.1.3
madmom==0.16.dev0
music21==5.1.0
sharedarray==2.0.4
markdown==2.2.0
tensorflow-gpu==1.2.0

please upgrade library module version

Question : May i know where did you implement the chord-conditional Arrangement generator??

Can you point me to the line where you implemented the Feature Extract and chord-conditioned Arrangement-Sheet generator? Thanks !
ps. point the data pipeline of lead sheet to chord/melody and feature extraction implementation.

fatal error C1083: Cannot open include file: 'sys/mman.h': No such file or directory

Hey! I installed LeadsheetArrangement in a env of python3.6 by the miniconda in my x64 win10. However, when i ran "pip install -r requirement.txt", some error happened:
fatal error C1083: Cannot open include file: 'sys/mman.h': No such file or directory
Can you tell me how to fix it? Thx!

OSError: [Errno 29] Illegal seek: 'val_X_bars'

when run scripts.py, meet the error like:OSError: [Errno 29] Illegal seek: 'val_X_bars'. But when create this variable, output :val_X_bars exsits. So how can I solve this problem? Thanks~

training data

Would you minded providing us with the training data?

pytorch implementation

I made a pytorch implementation of this (only for chroma_sequence and chord_sequence). If anyone interested welcome to check out my code. thanks for the great idea behind this.

Why chord-roll condition have height of 84

Hi Hau-min:
I'm trying to implement a pytorch version of LeadsheetArrangement. I'm wondering why the chord-roll condition has 84 height since only 12 pitch classes can have the equal information. Is it because it's good to keep the chord-roll condition the same size as the data?