bytedance / giantmidi-piano Goto Github PK

Python 95.49% Shell 4.51%

giantmidi-piano's Introduction

GiantMIDI-Piano

GiantMIDI-Piano [1] is a classical piano MIDI dataset contains 10,855 MIDI files of 2,786 composers. The curated subset by constraining composer surnames contains 7,236 MIDI files of 1,787 composers. GiantMIDI-Piano are transcribed from live recordings with a high-resolution piano transcription system [2].

Here is the demo of GiantMIDI-Piano: https://www.youtube.com/watch?v=5U-WL0QvKCg

Transcribed MIDI files of GiantMIDI-Piano can be viewed at midis_preview directory.

Download GiantMIDI-Piano

Method 1 (suggested)

Follow disclaimer.md to agree a disclaimer and download a stable version of GiantMIDI-Piano (193 MB).

Method 2

Users can acquire GiantMIDI-Piano by downloading all audio recordings, and transcribing them into MIDI files following the rest part of this repo. The transcription takes ~200 hours on a single GPU card.

Install requirements

Install PyTorch (>=1.4) following https://pytorch.org/.

The above links also include a curated subset. The curated subset constrains the YouTube titles should contain composers surnames.

pip install -r requirements.txt

Download audio recordings

Download audio recordings from YouTube using the following scripts. Approximately 10,855 audio recordings can be downloaded. There can be audios no longer downloadable.

WORKSPACE="./workspace"
mkdir -p $WORKSPACE
cp "resources/full_music_pieces_youtube_similarity_pianosoloprob.csv" $WORKSPACE/"full_music_pieces_youtube_similarity_pianosoloprob.csv"

# Download all mp3s. Users could split the downloading into parts to speed up the downloading. E.g.,
python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=0 --end_index=30000
python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=30000 --end_index=60000
python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=60000 --end_index=90000
python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=90000 --end_index=120000
python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=12000 --end_index=150000

The downloaded mp3 files look like:

mp3s_piano_solo (10,855 files)
├── Aaron, Michael, Piano Course, V8WvKK-1b2c.mp3
├── Aarons, Alfred E., Brother Bill, Giet2Krl6Ww.mp3
└── ...

Transcribe audios to MIDI files

# Transcribe all mp3s to midi files. Users could split the transcription into parts to speed up the transcription. E.g.,
python3 audios_to_midis.py transcribe_piano --workspace=$WORKSPACE --mp3s_dir=$WORKSPACE"/mp3s_piano_solo" --midis_dir=$WORKSPACE"/midis" --begin_ind=0 --end_index=30000
python3 audios_to_midis.py transcribe_piano --workspace=$WORKSPACE --mp3s_dir=$WORKSPACE"/mp3s_piano_solo" --midis_dir=$WORKSPACE"/midis" --begin_ind=30000 --end_index=60000
python3 audios_to_midis.py transcribe_piano --workspace=$WORKSPACE --mp3s_dir=$WORKSPACE"/mp3s_piano_solo" --midis_dir=$WORKSPACE"/midis" --begin_ind=60000 --end_index=90000
python3 audios_to_midis.py transcribe_piano --workspace=$WORKSPACE --mp3s_dir=$WORKSPACE"/mp3s_piano_solo" --midis_dir=$WORKSPACE"/midis" --begin_ind=90000 --end_index=120000
python3 audios_to_midis.py transcribe_piano --workspace=$WORKSPACE --mp3s_dir=$WORKSPACE"/mp3s_piano_solo" --midis_dir=$WORKSPACE"/midis" --begin_ind=120000 --end_index=150000

The transcribed MIDI files look like:

midis (10,855 files)
├── Aaron, Michael, Piano Course, V8WvKK-1b2c.mid
├── Abel, Frederic, Lola Polka, SLNJF0uiqRw.mid
└── ...

The transcription of all audio recordings may take around 10 days on a single GPU card.

Details of scripts can be viewed at scripts

Analyses the statistics of GiantMIDI-Piano

All statistics and figures in [1] can be reproduced by:

./scripts/3_statistics.sh

FAQ

If users met "Too many requests! Sleep for 3600 s" when downloading, it means that YouTube has limited the number of videos for downloading. Users could either 1) Wait until YouTube unblock your IP (1 days or a few weeks), or 2) try to use another machine with a different IP for downloading.

Contact

Qiuqiang Kong, [email protected]

Cite

[1] Qiuqiang Kong, Bochen Li, Jitong Chen, and Yuxuan Wang. "GiantMIDI-Piano: A large-scale MIDI dataset for classical piano music." arXiv preprint arXiv:2010.07061 (2020). https://arxiv.org/pdf/2010.07061

License

CC BY 4.0

giantmidi-piano's People

Contributors

Stargazers

Watchers

Forkers

xinaction laosuan aozhi lizhengdao zhangaustin cwb96 zyh1234 w84549275 ericlrf tianchi03 animfanis123 manbaum ansonting zhuyawen a350-1000 maclvern ffmryan the-xiongdanyu asukaleo-aa vomyrak zhang-ray xiaxzp vertonghen waynebaby boluoyu caerulea-rosa assassindesign aierlma efandong mufeng1991 husin123 sunnyscenery zhuchance lhcui asdlei99 verlaski devileo qinyanchen ai-project-collection jiubook ceshijivco stjordanis insmoin xmyhhh windbise noise-labs ksmos wantee asigalov61 tobeperson walnutpedia tawwxi jiangbin216 sydragon dnfcallan hotwindstyle lchoseni rexiyz xingfanxia e7mac katie-cathy-hunt hedgerealty wentaohub gyf513 yuzheng-chen sxc3202 xc1984759471 michealsnn jackymardt mengzifds brisk-calorie p0etry-rain hmrg-grmh choge543 vocaliodmiku cfuncode mn3mos jojoxiaok pangkaiguo leizton davinvlee rodcanada megatronx6 johnson7788 casera-meow superfzx ashinzeng namimono flaing zhangyong201911 hhh-0 stevenjokess bensive georgehjj7 ew232 110816008 zeromoon7 ioying yeliu0218 lindasilk

giantmidi-piano's Issues

Error when processing stereo audio

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-6634fd8f8eb4> in <module>
----> 1 dict = tran.transcribe(audio, "Ascended Vibrations.mid")

A:\***\piano_transcription_inference\inference.py in transcribe(self, audio, midi_path)
     80             * self.segment_samples - audio_len
     81
---> 82         audio = np.concatenate((audio, np.zeros((1, pad_len))), axis=1)
     83
     84         # Enframe to segments

<__array_function__ internals> in concatenate(*args, **kwargs)

ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 1 has 2 dimension(s)

Error when downloading MP3 audio data

Hi!

Thanks for this amazing work.

I've encountered a small issue while downloading the audio data. My first impression is that it is related to files not being available anymore. Here is the log:

python3 dataset.py download_youtube_piano_solo --workspace=$WORKSPACE --begin_index=0 --end_index=30000
[nltk_data] Downloading package punkt to /home/aferro/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/aferro/nltk_data...
[nltk_data]   Package averaged_perceptron_tagger is already up-to-
[nltk_data]       date!
0; Jag A.; Je t'aime Juliette; Je t'aime Juliette - A. Jag
ERROR: Video unavailable

[]
1; C. A. Aadler; Floating Islands; Mind-Boggling Off-Grid FLOATING Island HOMESTEAD
Traceback (most recent call last):
  File "dataset.py", line 660, in <module>
    download_youtube_piano_solo(args)
  File "dataset.py", line 565, in download_youtube_piano_solo
    if float(meta_dict['piano_solo_prob'][n]) >= 0.5:
ValueError: could not convert string to float: ''

As we can see, the meta_dict['piano_solo_prob'][n] entry is expected to yield a string that can be casted to a float, i.e. something like "0.12345". But sometimes it yields empty strings, which cannot be casted into floats.

Without analyzing too much of the code, a possible fix could be the following:

try:
    prob = float(meta_dict['piano_solo_prob'][n])
except ValueError as ve:
    print("SKIPPING ENTRY DUE TO ERROR:", ve)
    n += 1
    continue

if prob >= 0.5:
    count += 1
    ...etc

So far this seems to run OK on my end, yielding the desired audio data and logs under "$WORKSPACE", but I'm not sure if we're supposed to ignore the empty n entries, or rather fix them so no empty entries are provided. What do you think?
If this looks OK feel free to use the code, or let me know if you'd like me to do a PR.

Cheers,
Andres

instal issue

It's VERY hard

怎么可以复现“High-Resolution Piano Transcription With Pedals by Regressing Onset and Offset Times”这篇论文的实现呢

可以处理人声吗？

能不能做一个识别吉他的ai？

类似这个识别钢琴的

Illegal instruction

如图
为了在安卓上运行它，我装了linux容器并装好了所有依赖
但是我第一次运行时它说

Error in cpuinfo: failed to parse the list of possible processors in /sys/devices/system/cpu/possible Error in cpuinfo: failed to parse the list of present processors in /sys/devices/system/cpu/present Error in cpuinfo: failed to parse both lists of possible and present processors
因为linux容器本来就没这两个文件于是我手动创建了它们然后编好了内容
于是再次运行时就出现了Illegal instruction
可能是安卓特色(?)
有啥解决方案没？