zfturbo / music-source-separation-training Goto Github PK
View Code? Open in Web Editor NEWRepository for training models for music source separation.
Repository for training models for music source separation.
I am able to download the weights but there is no config file link it just takes me back to here
@ZFTurbo, Привет еще раз, заметил что Вы из России/Казахстана, приятно видеть это)
У меня появилась идея, я портировал Ваш скрипт на Google Kaggle (выложу его на GitHub чуть позже), как думаете, возможно ли ускорить тренировку модели путем запуска алгоритма на нескольких серверах?
Представим, что у нас есть 4 бесплатных аккаунта Google Collab с GPU T4. Было бы здорово если можно на 4 аккаунтах запустить скрипт тренировки, и Google Коллабы обменивались через DDP (PyTorch Distributed Data Parallel) или Horovod (фреймворк, разработанный Uber, для распределенного обучения моделей глубокого обучения).
Это сильно ускорит тренировку моделей, и выйдет бесплатно, чем аренда сервера с 8xTESLA T4.
Я готов написать код который будет запускать на разных Google Collab аккаунтах скрипт. Нужно лишь связать сервера.
Я не знаю тонкостей обучения моделей, хочу послушать Ваше мнение на счет этого, еще раз спасибо за большой вклад в сферу нейросетей и музыки.
Также подскажите пожалуйста кол-во треков в Вашем датасете для mdx23c. Я нашел отличный ресурс где можно получить оригинальные стемсы популярных треков студийного качества. https://songstems.net/ Возможно это будет полезно для вас) Хотелось бы собрать датасет но возможно будет проблема с загрузкой его куда-либо, авторские права и DMCA как никак(
Есть возможность, получить ваш Телеграм?
Hi author, I used htdemucs to train musdb18hq dataset, and trained more than 700 epochs why the vocals result is still only 4.56db
Hello, I have the seperation training setup properly and am able to utilize which model I choose. This is great, but I see that the leader boards have first place using MDX23C and BS Roformer (https://mvsep.com/quality_checker/entry/6240).
I've tried but can't get multiple models to be used at once - is there a way for us to use two models in an ensemble mode at once like the top of the leaderboards did using this separation training script?
Thanks so much!
Hi @ZFTurbo. First of all, thank you very much for this repo, well beyond my expectations (bs-roformer training) with the other models.
I use the config_musdb18_mel_band_roformer.yaml
with only two parameters changes: batch_size: 2
and target_instrument: drums
.
------------------------------------------
Epoch | Train loss | SDR drums
------------------------------------------
0 | 2.016814 | 2.3636
1 | 1.836035 | 2.7670
2 | 1.768249 | 3.6295
3 | 1.754987 | 3.6963
4 | 1.698568 | 3.4976
5 | 1.665295 | 4.1241
6 | 1.657550 | 4.4150
7 | 1.617158 | 4.4690
8 | 1.600137 | 4.5702
9 | 1.594573 | 4.6221
10 | 1.522084 | 4.8008
11 | 1.524944 | 4.9101
12 | 1.504670 | 5.0437
13 | 1.478117 | 5.1029
14 | ........ | ......
------------------------------------------
Mean Train Time (1000 steps / epoch) : 4 min 20 sec
Mean Valid Time ( 50 steps / epoch) : 8 min 00 sec
EDIT: with a batch_size of 1, I have a better training time : ~ 2 min 30.
Probably a question of low VRAM size. The valid time stay the same.
CPU : 13th Gen Intel® Core™ i9-13900K
GPU : NVIDIA GeForce RTX 4080 (16GB)
RAM : 64GB
SSD : Kingston FURY Renegade (2TB)
HDD : Seagate SkyHawk AI (16TB)
The training configuration is set to 1000 epochs
but I would like to know how many epoch you have to do to get the Vocal SDR 8.42
on the pre-trained (same model)? Currently, for a maximum of 1000 epochs training, the whole duration on my settings will be : 1000 * 12 min 20 sec, so 740000 sec / ~ 12333 min / ~ 205 hours / ~ 8-9 days.
The current training dataset model is based on direct-from-disk loading for audio chunk. I heard the 'digging' of the reading at each step during training. The MUSDB18HQ is (only) 29GB of size for the whole dataset, so can fit on the RAM on most of modern computer? Don't you think that this could improve the performance of the training steps and that it could be an additional option?
Thank a lot for your answer!
Why not release the weights in .onnx so we can use it in UVR directly? or is there any way to do this easily?
I tried to use a pretrained vocal model for bs roformer and this is the error i get, no idea how to fix it
PS C:\Users\matt\Downloads\Music-Source-Separation-Training-v.1.0.3\Music-Source-Separation-Training-v.1.0.3> python inference.py --model_type bs_roformer --config_path "C:\Users\matt\Downloads\model_bs_roformer_ep_317_sdr_12.9755.yaml.txt" --start_check_point
"C:\Users\matt\Downloads\model_bs_roformer_ep_317_sdr_12.9755 (1).ckpt" --input_folder
"C:\Users\matt\Downloads\input" --store_dir "C:\Users\matt\Downloads\Music" --device_ids 0
Instruments: ['vocals', 'other']
Traceback (most recent call last):
File "inference.py", line 99, in
proc_folder(None)
File "inference.py", line 75, in proc_folder
model = get_model_from_config(args.model_type, config)
File "C:\Users\matt\Downloads\Music-Source-Separation-Training-v.1.0.3\Music-Source-Separation-Training-v.1.0.3\utils.py", line 28, in get_model_from_config
model = BSRoformer(
File "<@beartype(models.bs_roformer.bs_roformer.BSRoformer.init) at 0x24e8d24f160>", line 98, in init
TypeError: init() got an unexpected keyword argument 'linear_transformer_depth'
PS C:\Users\matt\Downloads\Music-Source-Separation-Training-v.1.0.3\Music-Source-Separation-Training-v.1.0.3>
Hi,
This is a really great repo. Can you share a sample colab notebook, that we can link with drive and train on different custom datasets? Also, I am a newbie in this domain. What should be the folder structure?
I have two folders
This may not be an actual issue, but my training method. Would like thoughts on my application.
I'm using the starting weight from mdx23c and the vocals config file.
I have ~300 tracks that I'm training with and 5 for testing. Which takes about 30 minutes per epoch on my 12 GB card.
I used to have the datasets split 80% (215 tracks) training and 20% (85 tracks) testing which would take over an hour just for the testing part.
SDR for the 80-20 dataset were Avg 13.8
SDR for the reduced testing are Avg 14.9
However, after the first training epoch any subsequent epoch the SDR stagnates in both training methods. And since my card has 12GB I can only afford to batch train on a single song, not the 6 for the project configured 48 GB memory cards. This is the only change I made to the configure file.
`audio:
chunk_size: 261120
dim_f: 4096
dim_t: 256
hop_length: 1024
n_fft: 8192
num_channels: 2
sample_rate: 44100
min_mean_abs: 0.001
model:
act: gelu
bottleneck_factor: 4
growth: 128
norm: InstanceNorm
num_blocks_per_scale: 2
num_channels: 128
num_scales: 5
num_subbands: 4
scale:
training:
batch_size: 1
gradient_accumulation_steps: 1
grad_clip: 0
instruments:
inference:
batch_size: 1
dim_t: 256
num_overlap: 4`
What might explain the stagnating SDR? Could the batch size configuration be to blame or testing size? Would restarting the training after epoch 0 be advisable since it does improve the SDR after the first epoch, but perhaps it's overtraining then?
I am still a beginner in programming.
But I would love to use this wonderful program of yours.
However, after downloading it, I have all the necessary files and when I run the code, I get an error.
I don't know how to solve it for me.
Could you please tell me what is wrong?
This programming of yours is going to be a big dream come true for me.
Thanks.
(base) C:\Users\User\Desktop\Music-Source-Separation-Training-main>python train.py
Traceback (most recent call last):
File "C:\Users\User\Desktop\Music-Source-Separation-Training-main\train.py", line 515, in
train_model(None)
File "C:\Users\User\Desktop\Music-Source-Separation-Training-main\train.py", line 332, in train_model
model, config = get_model_from_config(args.model_type, args.config_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\Desktop\Music-Source-Separation-Training-main\utils.py", line 14, in get_model_from_config
with open(config_path) as f:
^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
There's a lot of similarity between enhancement and deverberation models with source separation. They can use the same backbones, etc.
Really the only difference is that you are outputting only a single channel. The noise or reverberation you remove isn't good to minimize loss over, only the clean signal.
It would be great if this repo supported single channel output datasets!
(And, you could consider adding speech enhancement backbones like sepformer from speechbrain. This new diffusion based model, SGMSE has good pretrained checkpoints that would probably be good after finetuning for separating vocals from non vocals, assuming the new finetuning included reverberated vocals.)
Hello, author, why is the SDR the same for every epoch after 60 epochs of training using the musdb18hq dataset, and the training loss remains 0 all the time? I used the following command:
py -m inference.py --model swin_upernet \
--config_path config_vocals_swin_upernet.yaml \
--start_check_point /models/MSST/model_swin_upernet_ep_56_sdr_10.6703.ckpt ...
config downloaded from releases.
Failed to import transformers.models.upernet.modeling_upernet because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'
split_torch_state_dict_into_shards
is included starting from huggingface-hub version 0.23.0.
tokenizers 0.14.0 and 0.14.1 requires huggingface-hub<18.0
and transformers==4.35.0 requires tokenizers=0.14.*
Installed by changing requirements.txt:
huggingface-hub>=0.23
transformers~=4.35.0
But:
ERROR: Could not find a version that satisfies the requirement pedalboard==0.8.1 (from versions: 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9, 0.9.10, 0.9.11, 0.9.12)
ERROR: No matching distribution found for pedalboard==0.8.1
So:
pedalboard~=0.8.1
Then:
--> 110 with torch.amp.autocast(enabled=config.training.use_amp):
111 with torch.inference_mode():
112 if config.training.target_instrument is not None:
File c:\apps\MSST\Lib\site-packages\ml_collections\config_dict\config_dict.py:829, in ConfigDict.__getattr__(self, attribute)
827 return self[attribute]
828 except KeyError as e:
--> 829 raise AttributeError(e)
AttributeError: "'use_amp'"
So, a few errors later, utils.py:
- with torch.cuda.amp.autocast(enabled=config.training.use_amp):
+ with torch.amp.autocast(device):
Then:
ValueError: Make sure that the channel dimension of the pixel values match with the one set in the configuration.
So, I looked at:
if len(batch_data) >= batch_size or (i >= mix.shape[1]):
arr = torch.stack(batch_data, dim=0)
arr.shape = torch.Size([1, 2, 261632])
I was confused, so mix.shape = torch.Size([2, 3038448]). I didn't think it would work, but I tried concatenate:
arr = torch.cat(batch_data, dim=0)
arr.shape = torch.Size([2, 261632])
The value error went away, but:
[177](file:///C:/apps/MSST/models/upernet_swin_transformers.py:177) def cac2cws(self, x):
[178](file:///C:/apps/MSST/models/upernet_swin_transformers.py:178) k = self.num_subbands
--> [179](file:///C:/apps/MSST/models/upernet_swin_transformers.py:179) b, c, f, t = x.shape
[180](file:///C:/apps/MSST/models/upernet_swin_transformers.py:180) x = x.reshape(b, c, k, f // k, t)
[181](file:///C:/apps/MSST/models/upernet_swin_transformers.py:181) x = x.reshape(b, c * k, f // k, t)
ValueError: not enough values to unpack (expected 4, got 3)
I'm out of ideas now.
python inference.py \
--model_type mdx23c
--config_path configs/config_mdx23c_musdb18.yaml
--start_check_point results/last_mdx23c.ckpt
--input_folder input/wavs/
--store_dir separation_results/
--extract_instrumental ??
How should I write the extract_instrumental field?
it really slows down training because it just stops using the gpu randomly and then starts again...
Config:
audio:
chunk_size: 261632
dim_f: 4096
dim_t: 512
hop_length: 512
n_fft: 8192
num_channels: 2
sample_rate: 44100
min_mean_abs: 0.001
model:
encoder_name: resnet152 # look with torchseg.list_encoders(). Currently 858 available
decoder_type: fpn # unet, fpn
act: gelu
num_channels: 128
num_subbands: 8
training:
batch_size: 12
gradient_accumulation_steps: 1
grad_clip: 0
instruments:
- vocals
- bass
- drums
- other
lr: 5.0e-05
patience: 3
reduce_factor: 0.95
target_instrument: null
num_epochs: 1000
num_steps: 2000
q: 0.95
coarse_loss_clip: true
ema_momentum: 0.999
optimizer: adamw
other_fix: false # it's needed for checking on multisong dataset if other is actually instrumental
use_amp: true # enable or disable usage of mixed precision (float16) - usually it must be true
augmentations:
enable: true # enable or disable all augmentations (to fast disable if needed)
loudness: true # randomly change loudness of each stem on the range (loudness_min; loudness_max)
loudness_min: 0.5
loudness_max: 1.5
mixup: true # mix several stems of same type with some probability (only works for dataset types: 1, 2, 3)
mixup_probs: !!python/tuple # 2 additional stems of the same type (1st with prob 0.2, 2nd with prob 0.02)
- 0.2
- 0.02
mixup_loudness_min: 0.5
mixup_loudness_max: 1.5
# apply mp3 compression to mixture only (emulate downloading mp3 from internet)
mp3_compression_on_mixture: 0.01
mp3_compression_on_mixture_bitrate_min: 32
mp3_compression_on_mixture_bitrate_max: 320
mp3_compression_on_mixture_backend: "lameenc"
all:
channel_shuffle: 0.5 # Set 0 or lower to disable
random_inverse: 0.1 # inverse track (better lower probability)
random_polarity: 0.5 # polarity change (multiply waveform to -1)
mp3_compression: 0.01
mp3_compression_min_bitrate: 32
mp3_compression_max_bitrate: 320
mp3_compression_backend: "lameenc"
vocals:
pitch_shift: 0.1
pitch_shift_min_semitones: -5
pitch_shift_max_semitones: 5
seven_band_parametric_eq: 0.25
seven_band_parametric_eq_min_gain_db: -9
seven_band_parametric_eq_max_gain_db: 9
tanh_distortion: 0.1
tanh_distortion_min: 0.1
tanh_distortion_max: 0.7
other:
pitch_shift: 0.1
pitch_shift_min_semitones: -4
pitch_shift_max_semitones: 4
gaussian_noise: 0.1
gaussian_noise_min_amplitude: 0.001
gaussian_noise_max_amplitude: 0.015
time_stretch: 0.01
time_stretch_min_rate: 0.8
time_stretch_max_rate: 1.25
inference:
batch_size: 1
dim_t: 512
num_overlap: 1
Image of gpu use:
I was trying to test out the pre trained swin_upernet model you provided but uncountered the following:
Traceback (most recent call last):
File "E:\Music-Source-Separation-Training-main\Music-Source-Separation-Training-main\inference.py", line 99, in <module>
proc_folder(None)
File "E:\Music-Source-Separation-Training-main\Music-Source-Separation-Training-main\inference.py", line 95, in proc_folder
run_folder(model, args, config, device, verbose=False)
File "E:\Music-Source-Separation-Training-main\Music-Source-Separation-Training-main\inference.py", line 44, in run_folder
res = demix_track(config, model, mixture, device)
File "E:\Music-Source-Separation-Training-main\Music-Source-Separation-Training-main\utils.py", line 62, in demix_track
x = model(part.unsqueeze(0))[0]
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\Music-Source-Separation-Training-main\Music-Source-Separation-Training-main\models\upernet_swin_transformers.py", line 201, in forward
x = self.swin_upernet_model(x).logits
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\transformers\models\upernet\modeling_upernet.py", line 406, in forward
outputs = self.backbone.forward_with_filtered_kwargs(
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\transformers\utils\backbone_utils.py", line 210, in forward_with_filtered_kwargs
return self(*args, **filtered_kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\transformers\models\swin\modeling_swin.py", line 1313, in forward
embedding_output, input_dimensions = self.embeddings(pixel_values)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\transformers\models\swin\modeling_swin.py", line 263, in forward
embeddings, output_dimensions = self.patch_embeddings(pixel_values)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\wesle\AppData\Roaming\Python\Python39\site-packages\transformers\models\swin\modeling_swin.py", line 315, in forward
raise ValueError(
ValueError: Make sure that the channel dimension of the pixel values match with the one set in the configuration.
I've made no changes to the configs and have tried updating my packages but no luck.
for bs roformer model
Hi, I'm training a vocals_mdx23c so am using dataset 2 from my understanding
I'm a little confused, how do I set up the validation folder? The same as the training folder (training -> (vocals and other folders))
Regarding the ensemble script, an explanation of the types could be added, and also, when I use the “fft” types, the following error occurs
Traceback (most recent call last):
File "G:\ensemble\ensemble.py", line 162, in <module>
ensemble_files(None)
File "G:\ensemble\ensemble.py", line 156, in ensemble_files
res = average_waveforms(data, weights, args.type)
File "G:\ensemble\ensemble.py", line 89, in average_waveforms
spec = stft(pred_track[i], 2048, 1024)
File "G:\ensemble\ensemble.py", line 14, in stft
spec_left = librosa.stft(wave_left, nfft, hop_length=hl)
TypeError: stft() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given
With the following command
python ensemble.py --files "1_01. Wings of Reality_(Vocals).flac" "1_01. Wings of Reality_MDX23C-8KFFT-InstVoc_HQ_(Vocals).flac" --type avg_fft --weights 2 1
i am training mel_band_roformer and changed train.py line 500 if 0
to if 1
when starting SDR evaluation
File "train.py", line 517, in <module>
train_model(None)
File "train.py", line 501, in train_model
sdr_avg = valid(model, args, config, device, verbose=False)
File "train.py", line 136, in valid
res = demix_track(config, model, mixture, device)
File "utils.py", line 130, in demix_track
result[..., start:start+l] += x[j][..., :l].cpu() * window[..., :l]
RuntimeError: The size of tensor a (131418) must match the size of tensor b (131584) at non-singleton dimension 1
arr.size()
is torch.Size([4, 2, 131584])
x.size()
is torch.Size([4, 2, 131418])
Errors when running inference using the pre-trained MelBand RoFormer Vocal model and the associated Config file.
RuntimeError: Error(s) in loading state_dict for MelBandRoformer:
Missing key(s) in state_dict: "mask_estimators.0.to_freqs.0.0.4.weight", "mask_estimators.0.to_freqs.0.0.4.bias", ...
size mismatch for mask_estimators.0.to_freqs.0.0.2.weight: copying a param with shape torch.Size([56, 768]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for mask_estimators.0.to_freqs.0.0.2.bias: copying a param with shape torch.Size([56]) from checkpoint, the shape in current model is torch.Size([768]). ...
There are some differences between the file linked in readme and the file in the main branch, but the errors happen with either file.
I don't have much useful information to add, but I did make a Colab notebook to eliminate variables. You can use it, too. It takes about four minutes to install and produce the error. https://colab.research.google.com/drive/1XTTWrs-FJKotFYtH8goTpaT3lIOTKUv1?usp=sharing
Do you have any interest into making a setup.py or at least a top-level package?
I have been working with your code in a pipeline of my own. However, it's annoying to import the packages here because I already have a train and utils to import.
So import music_source_separation_training.train would be great!
To post your model, please, fill the form:
Description:
Instruments:
Dataset (if known):
Metrics (if known):
Config link:
Checkpoint link:
Hi,
What dataset you use for vocals_mdx23c model training ? I want to reproduce your training and based on which to make improvement.
I think there is a mistake here
model_bandit_plus_dnr_sdr_11.47.chpt
,it must be .ckpt
file not .chpt
file
am i right ?
I use MDX23C for training and did a test run of 80 epochs but after running the training script again it started at epoch 0 instead of continue training. My folders are on C:.
It also doesn't show the loss while training on CMD.
Also, is it possible that it saves a model when the sdr is better than before?
(Example: epoch 50 had the last best sdr but now epoch 60 has a better sdr than epoch 50 etc.)
We can try to gather all public and private datasets in one place. To post dataset, please, fill the form:
Dataset name:
Description:
Instuments:
Format: sample rate and compression
Volume: number of tracks
Size: in GB
Download link:
When I try training with a single GPU (RTX 6000 Ada), it runs fine, but if I try it with multiple GPUs (using the same config file), it gets stuck here:
Instruments: ['vocal', 'guitar', 'piano', 'drums', 'bass', 'synth']
Use augmentation for training
Loading songs data from cache: /workspace/results2/metadata_1.pkl. If you updated dataset remove metadata_1.pkl before training!
Found tracks in dataset: 15300
Use multi GPU: [0, 1, 2]
Patience: 2 Reduce factor: 0.95 Batch size: 12 Grad accum steps: 1 Effective batch size: 12
Train for: 1000
Train epoch: 0 Learning rate: 9e-05
0%| | 0/1000 [00:00<?, ?it/s]
I am calling the script like this:
!python Music-Source-Separation-Training/train.py
--model_type htdemucs
--config_path /workspace/config.yaml
--results_path /workspace/results/
--data_path /workspace/StemGMD/training_output
--valid_path /workspace/StemGMD/training_output_eval
--dataset_type 1
--num_workers 8
--device_ids 0 1 2
My config file:
audio:
chunk_size: 485100 # samplerate * segment
min_mean_abs: 0.000
hop_length: 1024
training:
batch_size: 4
gradient_accumulation_steps: 1
grad_clip: 0
segment: 11
shift: 1
samplerate: 44100
channels: 2
normalize: true
instruments: ['vocal', 'guitar', 'piano', 'drums', 'bass', 'synth']
target_instrument: null
num_epochs: 1000
num_steps: 1000
optimizer: adam
lr: 9.0e-05
patience: 2
reduce_factor: 0.95
q: 0.95
coarse_loss_clip: true
ema_momentum: 0.999
other_fix: false # it's needed for checking on multisong dataset if other is actually instrumental
use_amp: true # enable or disable usage of mixed precision (float16) - usually it must be true
augmentations:
enable: true # enable or disable all augmentations (to fast disable if needed)
loudness: true # randomly change loudness of each stem on the range (loudness_min; loudness_max)
loudness_min: 0.5
loudness_max: 1.5
inference:
num_overlap: 4
batch_size: 8
model: hdemucs
hdemucs: # see demucs/hdemucs.py for a detailed description
channels: 48
channels_time: null
growth: 2
nfft: 4096
wiener_iters: 0
end_iters: 0
wiener_residual: False
cac: True
depth: 6
rewrite: True
hybrid: True
hybrid_old: False
multi_freqs: []
multi_freqs_depth: 3
freq_emb: 0.2
emb_scale: 10
emb_smooth: True
kernel_size: 8
stride: 4
time_stride: 2
context: 1
context_enc: 0
norm_starts: 4
norm_groups: 4
dconv_mode: 1
dconv_depth: 2
dconv_comp: 4
dconv_attn: 4
dconv_lstm: 4
dconv_init: 0.001
rescale: 0.1
I'm running the following command on a server with 32GB of RAM, 8GB of swap, without a GPU and it's getting killed after a few seconds. It will fill both memory and swap completely before getting killed.
python3 inference.py --model_type htdemucs --config_path configs/config_musdb18_htdemucs.yaml --start_check_point checkpoints/htdemucs_ft_bass.th --input_folder /input --store_dir /output/
Is there a way to limit the amount of RAM being used ?
As I understand, you train vocal models on you internal dataset, but could you share the total length of audio duration? I want to understand what the dataset size is approximately needed to get such metrics.
Hi, I'm using windows + anaconda.
I've downloaded the bandit v2 model "checkpoint-eng.ckpt" from https://zenodo.org/records/12701995 (which was linked in the https://github.com/kwatcharasupat/bandit-v2 repository).
I've put the model in the models folder, but when I run the following command I'm getting an error:
(msst_2) C:\Users\dreiD\Documents\github\Music-Source-Separation-Training>python inference.py --model_type bandit_v2 --config_path configs\config_dnr_bandit_v2_mus64.yaml --start_check_point models\checkpoint-eng.ckpt --input_folder input --store_dir separation_results
Start from checkpoint: models\checkpoint-eng.ckpt
Traceback (most recent call last):
File "C:\Users\dreiD\Documents\github\Music-Source-Separation-Training\inference.py", line 129, in
proc_folder(None)
File "C:\Users\dreiD\Documents\github\Music-Source-Separation-Training\inference.py", line 109, in proc_folder
model.load_state_dict(state_dict)
File "C:\Users\dreiD\anaconda3\envs\msst_2\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Bandit:
Missing key(s) in state_dict: "stft.window", "istft.window", "band_split.norm_fc_modules.0.combined.0.weight", "band_split.norm_fc_modules.0.combined.0.bias", "band_split.norm_fc_modules.0.combined.1.weight", "band_split.norm_fc_modules.0.combined.1.bias", "band_split.norm_fc_modules.1.combined.0.weight", "band_split.norm_fc_modules.1.combined.0.bias", "band_split.norm_fc_modules.1.combined.1.weight", "band_split.norm_fc_modules.1.combined.1.bias", "band_split.norm_fc_modules.2.combined.0.weight", ....
...
"band_split.norm_fc_modules.63.combined.0.bias", "band_split.norm_fc_modules.63.combined.1.weight", "band_split.norm_fc_modules.63.combined.1.bias", "tf_model.seqband.0.norm.weight", "tf_model.seqband.0.norm.bias", "tf_model.seqband.0.rnn.weight_ih_l0", "tf_model.seqband.0.rnn.weight_hh_l0", "tf_model.seqband.0.rnn.bias_ih_l0", "tf_model.seqband.0.rnn.bias_hh_l0", "tf_model.seqband.0.rnn.weight_ih_l0_reverse", "tf_model.seqband.0.rnn.weight_hh_l0_reverse", "tf_model.seqband.0.rnn.bias_ih_l0_reverse", "tf_model.seqband.0.rnn.bias_hh_l0_reverse", "tf_model.seqband.0.fc.weight", "tf_model.seqband.0.fc.bias", "tf_model.seqband.2.norm.weight", "tf_model.seqband.2.norm.bias", "tf_model.seqband.2.rnn.weight_ih_l0", "tf_model.seqband.2.rnn.weight_hh_l0", "tf_model.seqband.2.rnn.bias_ih_l0", "tf_model.seqband.2.rnn.bias_hh_l0", "tf_model.seqband.2.rnn.weight_ih_l0_reverse", "tf_model.seqband.2.rnn.weight_hh_l0_reverse", "tf_model.seqband.2.rnn.bias_ih_l0_reverse", "tf_model.seqband.2.rnn.bias_hh_l0_reverse", "tf_model.seqband.2.fc.weight", "tf_model.seqband.2.fc.bias", "tf_model.seqband.4.norm.weight", "tf_model.seqband.4.norm.bias", "tf_model.seqband.4.rnn.weight_ih_l0", "tf_model.seqband.4.rnn.weight_hh_l0", "tf_model.seqband.4.rnn.bias_ih_l0", "tf_model.seqband.4.rnn.bias_hh_l0", "tf_model.seqband.4.rnn.weight_ih_l0_reverse", "tf_model.seqband.4.rnn.weight_hh_l0_reverse", "tf_model.seqband.4.rnn.bias_ih_l0_reverse", "tf_model.seqband.4.rnn.bias_hh_l0_reverse", "tf_model.seqband.4.fc.weight", "tf_model.seqband.4.fc.bias", "tf_model.seqband.6.norm.weight", "tf_model.seqband.6.norm.bias", "tf_model.seqband.6.rnn.weight_ih_l0", ...
"tf_model.seqband.30.rnn.weight_ih_l0", "tf_model.seqband.30.rnn.weight_hh_l0", "tf_model.seqband.30.rnn.bias_ih_l0", "tf_model.seqband.30.rnn.bias_hh_l0", "tf_model.seqband.30.rnn.weight_ih_l0_reverse", "tf_model.seqband.30.rnn.weight_hh_l0_reverse", "tf_model.seqband.30.rnn.bias_ih_l0_reverse", "tf_model.seqband.30.rnn.bias_hh_l0_reverse", "tf_model.seqband.30.fc.weight", "tf_model.seqband.30.fc.bias", "mask_estim.speech.freq_weights/0", "mask_estim.speech.freq_weights/1", "mask_estim.speech.freq_weights/2", "mask_estim.speech.freq_weights/3", "mask_estim.speech.freq_weights/4", "mask_estim.speech.freq_weights/5", "mask_estim.speech.freq_weights/6", "mask_estim.speech.freq_weights/7", "mask_estim.speech.freq_weights/8", "mask_estim.speech.freq_weights/9", "mask_estim.speech.freq_weights/10", "mask_estim.speech.freq_weights/11", "mask_estim.speech.freq_weights/12", "mask_estim.speech.freq_weights/13", "mask_estim.speech.freq_weights/14", "mask_estim.speech.freq_weights/15", "mask_estim.speech.freq_weights/16", "mask_estim.speech.freq_weights/17", "mask_estim.speech.freq_weights/18", "mask_estim.speech.freq_weights/19", "mask_estim.speech.freq_weights/20", "mask_estim.speech.freq_weights/21", "mask_estim.speech.freq_weights/22", "mask_estim.speech.freq_weights/23", "mask_estim.speech.freq_weights/24", "mask_estim.speech.freq_weights/25", "mask_estim.speech.freq_weights/26", "mask_estim.speech.freq_weights/27", "mask_estim.speech.freq_weights/28", "mask_estim.speech.freq_weights/29", "mask_estim.speech.freq_weights/30", "mask_estim.speech.freq_weights/31", "mask_estim.speech.freq_weights/32", "mask_estim.speech.freq_weights/33", "mask_estim.speech.freq_weights/34", "mask_estim.speech.freq_weights/35", "mask_estim.speech.freq_weights/36", "mask_estim.speech.freq_weights/37", "mask_estim.speech.freq_weights/38", "mask_estim.speech.freq_weights/39", "mask_estim.speech.freq_weights/40", "mask_estim.speech.freq_weights/41", "mask_estim.speech.freq_weights/42", "mask_estim.speech.freq_weights/43", "mask_estim.speech.freq_weights/44", "mask_estim.speech.freq_weights/45", "mask_estim.speech.freq_weights/46", "mask_estim.speech.freq_weights/47", "mask_estim.speech.freq_weights/48", "mask_estim.speech.freq_weights/49", "mask_estim.speech.freq_weights/50", "mask_estim.speech.freq_weights/51", "mask_estim.speech.freq_weights/52", "mask_estim.speech.freq_weights/53", "mask_estim.speech.freq_weights/54", "mask_estim.speech.freq_weights/55", "mask_estim.speech.freq_weights/56", "mask_estim.speech.freq_weights/57", "mask_estim.speech.freq_weights/58", "mask_estim.speech.freq_weights/59", "mask_estim.speech.freq_weights/60", "mask_estim.speech.freq_weights/61", "mask_estim.speech.freq_weights/62", "mask_estim.speech.freq_weights/63", "mask_estim.speech.norm_mlp.0.norm.weight", "mask_estim.speech.norm_mlp.0.norm.bias", "mask_estim.speech.norm_mlp.0.hidden.0.weight", "mask_estim.speech.norm_mlp.0.hidden.0.bias", "mask_estim.speech.norm_mlp.0.output.0.weight", "mask_estim.speech.norm_mlp.0.output.0.bias", "mask_estim.speech.norm_mlp.0.combined.0.weight", "mask_estim.speech.norm_mlp.0.combined.0.bias", "mask_estim.speech.norm_mlp.0.combined.1.0.weight", "mask_estim.speech.norm_mlp.0.combined.1.0.bias", "mask_estim.speech.norm_mlp.0.combined.2.0.weight", "mask_estim.speech.norm_mlp.0.combined.2.0.bias", "mask_estim.speech.norm_mlp.1.norm.weight", "mask_estim.speech.norm_mlp.1.norm.bias", "mask_estim.speech.norm_mlp.1.hidden.0.weight", "mask_estim.speech.norm_mlp.1.hidden.0.bias", "mask_estim.speech.norm_mlp.1.output.0.weight", "mask_estim.speech.norm_mlp.1.output.0.bias", "mask_estim.speech.norm_mlp.1.combined.0.weight", "mask_estim.speech.norm_mlp.1.combined.0.bias", "mask_estim.speech.norm_mlp.1.combined.1.0.weight", "mask_estim.speech.norm_mlp.1.combined.1.0.bias", "mask_estim.speech.norm_mlp.1.combined.2.0.weight", "mask_estim.speech.norm_mlp.1.combined.2.0.bias", "mask_estim.speech.norm_mlp.2.norm.weight", "mask_estim.speech.norm_mlp.2.norm.bias", "mask_estim.speech.norm_mlp.2.hidden.0.weight", "mask_estim.speech.norm_mlp.2.hidden.0.bias", "mask_estim.speech.norm_mlp.2.output.0.weight", "mask_estim.speech.norm_mlp.2.output.0.bias", "mask_estim.speech.norm_mlp.2.combined.0.weight", "mask_estim.speech.norm_mlp.2.combined.0.bias", "mask_estim.speech.norm_mlp.2.combined.1.0.weight", "mask_estim.speech.norm_mlp.2.combined.1.0.bias", "mask_estim.speech.norm_mlp.2.combined.2.0.weight", "mask_estim.speech.norm_mlp.2.combined.2.0.bias", "mask_estim.speech.norm_mlp.3.norm.weight", "mask_estim.speech.norm_mlp.3.norm.bias", "mask_estim.speech.norm_mlp.3.hidden.0.weight", ...
...
"mask_estim.speech.norm_mlp.63.norm.bias", "mask_estim.speech.norm_mlp.63.hidden.0.weight", "mask_estim.speech.norm_mlp.63.hidden.0.bias", "mask_estim.speech.norm_mlp.63.output.0.weight", "mask_estim.speech.norm_mlp.63.output.0.bias", "mask_estim.speech.norm_mlp.63.combined.0.weight", "mask_estim.speech.norm_mlp.63.combined.0.bias", "mask_estim.speech.norm_mlp.63.combined.1.0.weight", "mask_estim.speech.norm_mlp.63.combined.1.0.bias", "mask_estim.speech.norm_mlp.63.combined.2.0.weight", "mask_estim.speech.norm_mlp.63.combined.2.0.bias", "mask_estim.music.freq_weights/0", "mask_estim.music.freq_weights/1", "mask_estim.music.freq_weights/2", "mask_estim.music.freq_weights/3", "mask_estim.music.freq_weights/4", "mask_estim.music.freq_weights/5", "mask_estim.music.freq_weights/6", "mask_estim.music.freq_weights/7", "mask_estim.music.freq_weights/8", "mask_estim.music.freq_weights/9", "mask_estim.music.freq_weights/10", "mask_estim.music.freq_weights/11", "mask_estim.music.freq_weights/12", "mask_estim.music.freq_weights/13", "mask_estim.music.freq_weights/14", "mask_estim.music.freq_weights/15", "mask_estim.music.freq_weights/16", "mask_estim.music.freq_weights/17", "mask_estim.music.freq_weights/18", "mask_estim.music.freq_weights/19", "mask_estim.music.freq_weights/20", "mask_estim.music.freq_weights/21", "mask_estim.music.freq_weights/22", "mask_estim.music.freq_weights/23", "mask_estim.music.freq_weights/24", "mask_estim.music.freq_weights/25", "mask_estim.music.freq_weights/26", "mask_estim.music.freq_weights/27", "mask_estim.music.freq_weights/28", "mask_estim.music.freq_weights/29", "mask_estim.music.freq_weights/30", "mask_estim.music.freq_weights/31", "mask_estim.music.freq_weights/32", "mask_estim.music.freq_weights/33", "mask_estim.music.freq_weights/34", "mask_estim.music.freq_weights/35", "mask_estim.music.freq_weights/36", "mask_estim.music.freq_weights/37", "mask_estim.music.freq_weights/38", "mask_estim.music.freq_weights/39", "mask_estim.music.freq_weights/40", "mask_estim.music.freq_weights/41", "mask_estim.music.freq_weights/42", "mask_estim.music.freq_weights/43", "mask_estim.music.freq_weights/44", "mask_estim.music.freq_weights/45", "mask_estim.music.freq_weights/46", "mask_estim.music.freq_weights/47", "mask_estim.music.freq_weights/48", "mask_estim.music.freq_weights/49", "mask_estim.music.freq_weights/50", "mask_estim.music.freq_weights/51", "mask_estim.music.freq_weights/52", "mask_estim.music.freq_weights/53", "mask_estim.music.freq_weights/54", "mask_estim.music.freq_weights/55", "mask_estim.music.freq_weights/56", "mask_estim.music.freq_weights/57", "mask_estim.music.freq_weights/58", "mask_estim.music.freq_weights/59", "mask_estim.music.freq_weights/60", "mask_estim.music.freq_weights/61", "mask_estim.music.freq_weights/62", "mask_estim.music.freq_weights/63", "mask_estim.music.norm_mlp.0.norm.weight", "mask_estim.music.norm_mlp.0.norm.bias", "mask_estim.music.norm_mlp.0.hidden.0.weight", "mask_estim.music.norm_mlp.0.hidden.0.bias", "mask_estim.music.norm_mlp.0.output.0.weight", "mask_estim.music.norm_mlp.0.output.0.bias", "mask_estim.music.norm_mlp.0.combined.0.weight", "mask_estim.music.norm_mlp.0.combined.0.bias", "mask_estim.music.norm_mlp.0.combined.1.0.weight", "mask_estim.music.norm_mlp.0.combined.1.0.bias", "mask_estim.music.norm_mlp.0.combined.2.0.weight", "mask_estim.music.norm_mlp.0.combined.2.0.bias", "mask_estim.music.norm_mlp.1.norm.weight", "mask_estim.music.norm_mlp.1.norm.bias", "mask_estim.music.norm_mlp.1.hidden.0.weight", "mask_estim.music.norm_mlp.1.hidden.0.bias", "mask_estim.music.norm_mlp.1.output.0.weight", "mask_estim.music.norm_mlp.1.output.0.bias", "mask_estim.music.norm_mlp.1.combined.0.weight", "mask_estim.music.norm_mlp.1.combined.0.bias", "mask_estim.music.norm_mlp.1.combined.1.0.weight", "mask_estim.music.norm_mlp.1.combined.1.0.bias", "mask_estim.music.norm_mlp.1.combined.2.0.weight", "mask_estim.music.norm_mlp.1.combined.2.0.bias", "mask_estim.music.norm_mlp.2.norm.weight", "mask_estim.music.norm_mlp.2.norm.bias", "mask_estim.music.norm_mlp.2.hidden.0.weight", "mask_estim.music.norm_mlp.2.hidden.0.bias", "mask_estim.music.norm_mlp.2.output.0.weight", "mask_estim.music.norm_mlp.2.output.0.bias", "mask_estim.music.norm_mlp.2.combined.0.weight", "mask_estim.music.norm_mlp.2.combined.0.bias", "mask_estim.music.norm_mlp.2.combined.1.0.weight", "mask_estim.music.norm_mlp.2.combined.1.0.bias", "mask_estim.music.norm_mlp.2.combined.2.0.weight", "mask_estim.music.norm_mlp.2.combined.2.0.bias", "mask_estim.music.norm_mlp.3.norm.weight", "mask_estim.music.norm_mlp.3.norm.bias", "mask_estim.music.norm_mlp.3.hidden.0.weight", "mask_estim.music.norm_mlp.3.hidden.0.bias", "mask_estim.music.norm_mlp.3.output.0.weight", "mask_estim.music.norm_mlp.3.output.0.bias", "mask_estim.music.norm_mlp.3.combined.0.weight", "mask_estim.music.norm_mlp.3.combined.0.bias", "mask_estim.music.norm_mlp.3.combined.1.0.weight", "mask_estim.music.norm_mlp.3.combined.1.0.bias", "mask_estim.music.norm_mlp.3.combined.2.0.weight", "mask_estim.music.norm_mlp.3.combined.2.0.bias", "mask_estim.music.norm_mlp.4.norm.weight", ...
...
"mask_estim.music.norm_mlp.63.norm.bias", "mask_estim.music.norm_mlp.63.hidden.0.weight", "mask_estim.music.norm_mlp.63.hidden.0.bias", "mask_estim.music.norm_mlp.63.output.0.weight", "mask_estim.music.norm_mlp.63.output.0.bias", "mask_estim.music.norm_mlp.63.combined.0.weight", "mask_estim.music.norm_mlp.63.combined.0.bias", "mask_estim.music.norm_mlp.63.combined.1.0.weight", "mask_estim.music.norm_mlp.63.combined.1.0.bias", "mask_estim.music.norm_mlp.63.combined.2.0.weight", "mask_estim.music.norm_mlp.63.combined.2.0.bias", "mask_estim.sfx.freq_weights/0", "mask_estim.sfx.freq_weights/1", "mask_estim.sfx.freq_weights/2", "mask_estim.sfx.freq_weights/3", "mask_estim.sfx.freq_weights/4", "mask_estim.sfx.freq_weights/5", "mask_estim.sfx.freq_weights/6", "mask_estim.sfx.freq_weights/7", "mask_estim.sfx.freq_weights/8", "mask_estim.sfx.freq_weights/9", "mask_estim.sfx.freq_weights/10", "mask_estim.sfx.freq_weights/11", "mask_estim.sfx.freq_weights/12", "mask_estim.sfx.freq_weights/13", "mask_estim.sfx.freq_weights/14", "mask_estim.sfx.freq_weights/15", "mask_estim.sfx.freq_weights/16", "mask_estim.sfx.freq_weights/17", "mask_estim.sfx.freq_weights/18", "mask_estim.sfx.freq_weights/19", "mask_estim.sfx.freq_weights/20", "mask_estim.sfx.freq_weights/21", "mask_estim.sfx.freq_weights/22", "mask_estim.sfx.freq_weights/23", "mask_estim.sfx.freq_weights/24", "mask_estim.sfx.freq_weights/25", "mask_estim.sfx.freq_weights/26", "mask_estim.sfx.freq_weights/27", "mask_estim.sfx.freq_weights/28", "mask_estim.sfx.freq_weights/29", "mask_estim.sfx.freq_weights/30", "mask_estim.sfx.freq_weights/31", "mask_estim.sfx.freq_weights/32", "mask_estim.sfx.freq_weights/33", "mask_estim.sfx.freq_weights/34", "mask_estim.sfx.freq_weights/35", "mask_estim.sfx.freq_weights/36", "mask_estim.sfx.freq_weights/37", "mask_estim.sfx.freq_weights/38", "mask_estim.sfx.freq_weights/39", "mask_estim.sfx.freq_weights/40", "mask_estim.sfx.freq_weights/41", "mask_estim.sfx.freq_weights/42", "mask_estim.sfx.freq_weights/43", "mask_estim.sfx.freq_weights/44", "mask_estim.sfx.freq_weights/45", "mask_estim.sfx.freq_weights/46", "mask_estim.sfx.freq_weights/47", "mask_estim.sfx.freq_weights/48", "mask_estim.sfx.freq_weights/49", "mask_estim.sfx.freq_weights/50", "mask_estim.sfx.freq_weights/51", "mask_estim.sfx.freq_weights/52", "mask_estim.sfx.freq_weights/53", "mask_estim.sfx.freq_weights/54", "mask_estim.sfx.freq_weights/55", "mask_estim.sfx.freq_weights/56", "mask_estim.sfx.freq_weights/57", "mask_estim.sfx.freq_weights/58", "mask_estim.sfx.freq_weights/59", "mask_estim.sfx.freq_weights/60", "mask_estim.sfx.freq_weights/61", "mask_estim.sfx.freq_weights/62", "mask_estim.sfx.freq_weights/63", "mask_estim.sfx.norm_mlp.0.norm.weight", "mask_estim.sfx.norm_mlp.0.norm.bias", "mask_estim.sfx.norm_mlp.0.hidden.0.weight", "mask_estim.sfx.norm_mlp.0.hidden.0.bias", "mask_estim.sfx.norm_mlp.0.output.0.weight", "mask_estim.sfx.norm_mlp.0.output.0.bias", "mask_estim.sfx.norm_mlp.0.combined.0.weight", "mask_estim.sfx.norm_mlp.0.combined.0.bias", "mask_estim.sfx.norm_mlp.0.combined.1.0.weight", "mask_estim.sfx.norm_mlp.0.combined.1.0.bias", "mask_estim.sfx.norm_mlp.0.combined.2.0.weight", "mask_estim.sfx.norm_mlp.0.combined.2.0.bias", "mask_estim.sfx.norm_mlp.1.norm.weight", "mask_estim.sfx.norm_mlp.1.norm.bias", "mask_estim.sfx.norm_mlp.1.hidden.0.weight", "mask_estim.sfx.norm_mlp.1.hidden.0.bias", "mask_estim.sfx.norm_mlp.1.output.0.weight", "mask_estim.sfx.norm_mlp.1.output.0.bias", "mask_estim.sfx.norm_mlp.1.combined.0.weight", "mask_estim.sfx.norm_mlp.1.combined.0.bias", "mask_estim.sfx.norm_mlp.1.combined.1.0.weight", "mask_estim.sfx.norm_mlp.1.combined.1.0.bias", "mask_estim.sfx.norm_mlp.1.combined.2.0.weight", "mask_estim.sfx.norm_mlp.1.combined.2.0.bias", "mask_estim.sfx.norm_mlp.2.norm.weight", "mask_estim.sfx.norm_mlp.2.norm.bias", "mask_estim.sfx.norm_mlp.2.hidden.0.weight", "mask_estim.sfx.norm_mlp.2.hidden.0.bias", "mask_estim.sfx.norm_mlp.2.output.0.weight", "mask_estim.sfx.norm_mlp.2.output.0.bias", "mask_estim.sfx.norm_mlp.2.combined.0.weight", "mask_estim.sfx.norm_mlp.2.combined.0.bias", "mask_estim.sfx.norm_mlp.2.combined.1.0.weight", "mask_estim.sfx.norm_mlp.2.combined.1.0.bias", "mask_estim.sfx.norm_mlp.2.combined.2.0.weight", "mask_estim.sfx.norm_mlp.2.combined.2.0.bias", "mask_estim.sfx.norm_mlp.3.norm.weight", "mask_estim.sfx.norm_mlp.3.norm.bias", "mask_estim.sfx.norm_mlp.3.hidden.0.weight", "mask_estim.sfx.norm_mlp.3.hidden.0.bias", "mask_estim.sfx.norm_mlp.3.output.0.weight", "mask_estim.sfx.norm_mlp.3.output.0.bias", "mask_estim.sfx.norm_mlp.3.combined.0.weight", "mask_estim.sfx.norm_mlp.3.combined.0.bias", "mask_estim.sfx.norm_mlp.3.combined.1.0.weight", "mask_estim.sfx.norm_mlp.3.combined.1.0.bias", "mask_estim.sfx.norm_mlp.3.combined.2.0.weight", "mask_estim.sfx.norm_mlp.3.combined.2.0.bias",
...
"mask_estim.sfx.norm_mlp.63.hidden.0.weight", "mask_estim.sfx.norm_mlp.63.hidden.0.bias", "mask_estim.sfx.norm_mlp.63.output.0.weight", "mask_estim.sfx.norm_mlp.63.output.0.bias", "mask_estim.sfx.norm_mlp.63.combined.0.weight", "mask_estim.sfx.norm_mlp.63.combined.0.bias", "mask_estim.sfx.norm_mlp.63.combined.1.0.weight", "mask_estim.sfx.norm_mlp.63.combined.1.0.bias", "mask_estim.sfx.norm_mlp.63.combined.2.0.weight", "mask_estim.sfx.norm_mlp.63.combined.2.0.bias".
Unexpected key(s) in state_dict: "epoch", "global_step", "pytorch-lightning_version", "state_dict", "loops", "callbacks", "optimizer_states", "lr_schedulers".
I've tried setting up a new conda env and re downloaded the repository but I'm getting the same error.
Other models like the bandit plus model did work.
Is roformer only configured for two stems (for example vocal and other)? I tried to run it with 4 stems (vocal / bass / guitar / drums) but it failed with a tensor size mismatch. Is there a config setting I'm missing? Thanks
Hi, i run the inference with your pretrained mdx23c model and get following errors:
(py3.10) python inference.py --model_type mdx23c \ [17:48:11]
--config_path configs/config_vocals_mdx23c.yaml \
--start_check_point pretrained/model_vocals_mdx23c_sdr_10.17.ckpt \
--input_folder /soundx_speech_data/eval_audio/singing \
--store_dir separation_results/
Instruments: ['vocals', 'other']
Start from checkpoint: pretrained/model_vocals_mdx23c_sdr_10.17.ckpt
Total tracks found: 7
0%| | 0/7 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/data1/martinzheng/tts/backend/Music-Source-Separation-Training/inference.py", line 99, in <module>
proc_folder(None)
File "/data1/martinzheng/tts/backend/Music-Source-Separation-Training/inference.py", line 95, in proc_folder
run_folder(model, args, config, device, verbose=False)
File "/data1/martinzheng/tts/backend/Music-Source-Separation-Training/inference.py", line 44, in run_folder
res = demix_track(config, model, mixture, device)
File "/data1/martinzheng/tts/backend/Music-Source-Separation-Training/utils.py", line 56, in demix_track
mix = nn.functional.pad(mix, (border, border), mode='reflect')
NotImplementedError: Only 2D, 3D, 4D, 5D padding with non-constant padding are supported for now
any ideas?
Hi, I am a beginner in machine learning. I want to try to train the MUSDB18HQ dataset with additional STEMS. Am I correct that I can do this and I don't need to download 20GB dataset?
I tried to run the training but got the answer that there are 0 tracks in the dataset. How many VALID (TEST) tracks would you recommend to upload in relation to the whole dataset, 20% is ok?
test/valid tracks:
Hi there, 1st of all thanks for this awesome work !
Since we've 'doxed' it in our HyMPS project (under AUDIO section \ AI-based page \ Source Separation), can you please add a GH-compliant license file for it ?
As you know, expliciting licensing terms is extremely important to let anyone better/faster understand how to reuse/adapt/modify sources (and not only) in other open projects and vice-versa.
Although it may sounds like a minor aspect, license file omission obviously causes an inconsistent generation of the relative badge too:
(badge-generator URL: https://badgen.net/github/license/ZFTurbo/Music-Source-Separation-Training)
You can easily set a standardized one through the GH's license wizard tool.
Last but not least, let us know how we could improve - in your opinion - our categorizations and links to resources in order to favor collaboration between developers (and therefore evolution) of listed projects.
Hope that helps/inspires !
I got the following error. Please let me know how to resolve it.
Traceback (most recent call last):
File "C:\Users\Users\desktop\Music-Source-Separation-Training-main\train.py", line 515, in
train_model(None)
File "C:\Users\Users\desktop\Music-Source-Separation-Training-main\train.py", line 332, in train_model
model, config = get_model_from_config(args.model_type, args.config_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Users\desktop\Music-Source-Separation-Training-main\utils.py", line 24, in get_model_from_config
from models.demucs4ht import get_model
File "C:\Users\Users\desktop\Music-Source-Separation-Training-main\models\demucs4ht.py", line 10, in
from demucs.demucs import Demucs
ModuleNotFoundError: No module named 'demucs'
It seems to reset to 0 every time
Hi,
First, thanks for the code.
If I understand correctly, creating appropriate train/val datasets require to provide a dataset_type
(for training songs) and valid_path
(for validation songs). In other words, the expected folder structure is (e.g., for a dataset type 1 such as MUSDB):
dataset/train/song 1/{ vocals.wav, bass.wav, drums.wav, other.wav}
dataset/train/song 2/{ vocals.wav, bass.wav, drums.wav, other.wav}
...........
dataset/valid/song 1/{ vocals.wav, bass.wav, drums.wav, other.wav}
dataset/valid/song 2/{ vocals.wav, bass.wav, drums.wav, other.wav}
...........
dataset/test/song 1/{ vocals.wav, bass.wav, drums.wav, other.wav}
dataset/test/song 2/{ vocals.wav, bass.wav, drums.wav, other.wav}
...........
However, this is not the structure of MUSDB, which only contains a train
and test
subfolders: the songs used for validation are part of the train
folder, and a fixed list of validation tracks is provided by the authors for reproducibility.
Therefore, in order to reproduce some papers' results using MUSDB18 (e.g., demucs, BSRNN), should we pre-process the dataset by manually creating a folder split? Or is there something I missed?
Thanks.
Paul
PS: one additional minor thing, I think an augmentation.mixup=True/False
entry should be added in the config_musdb18_mel_band_roformer
, config_musdb18_htdemucs
, and config_musdb18_bs_roformer
files, otherwise there's a "Missing key" error when instantiating these.
Hello, I am currently analyzing your code in an attempt to reproduce the paper performance of bs_roformer. While examining the code, I have come across a few points that I am curious about and would like to inquire further.
Upon reviewing the settings related to bs_roformer, I noticed that the configuration predominantly uses only vocals and other in the instruments setup. Generally, MSS datasets like MUSDB18 are composed of a 4-stem setup: [vocals, drums, bass, other], and the mixture is a combination of these four stems. However, your default setting is [vocals, other], and using this setting results in a mixture composed only of vocals + other. I am curious whether this configuration is an error or if there was a specific task intended for this particular setup.
Additionally, while examining the code, I noticed that loudness augmentation is applied once during the load_random_mix function when performing mixup, and again in getitem. I would like to clarify whether applying loudness augmentation twice is by design, a misunderstanding on my part, or a coding error.
I would appreciate your response to these two questions.
Thank you.
Thank you @ZFTurbo for providing the training code. I have tested out UVR MDX-Net models and found them to be very good. However, i am unable to get the models to work in real time. I wonder if the models provided here are capable of vocal separation in real time if i were to train them, or do i have to make some kind of adjustments to their architecture?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.