seungheondoh / emopia_cls Goto Github PK

Hi,
I want to reimplement the audio classification part.
When I used preprocessing.py , I found that I had to define where were audio files, but I only found youtube URL links in EMOPIA.zip.
As I knew, songs were separated into clips.
If I downloads audio files from youtube URL, how to divide them into clips?
thanks!

I try to download the source mp4 from Q1.txt, Q2.txt, Q3.txt, Q4.txt of songs_lists directory in the EMOPIA_2.2 datasets,
using the following scripts:

#!/bin/bash
input="$1"
root=`echo $input|cut -d "." -f 1`
count=1
while IFS= read -r line
do
  echo "Start download $count-th $line"
  output=`echo $line|cut -d "=" -f 2|cut -d "," -f 1`
  youtube-dl -x --audio-format mp3 -o "$root/$output.%(ext)s" $line
  count=$[count+1]
  sleep 3
done < "$input"

But failed to download some videos.

Start download 3-th https://www.youtube.com/watch?v=JP3QKZlyQz4
[youtube] JP3QKZlyQz4: Downloading webpage
ERROR: Video unavailable
This video is not available

@seungheondoh Could you share the source music datasets of EMOPIA_2.2 datasets, so that we can apply scripts/timestamp2clip.py to get get audio clips?

sir, I'm trying to train the wav domain classification model from scratch. When I run with python train_test.py --wav sr22k --task ar_va, an error is reported: IndexError: single positional indexer is out-of-bounds.

So I navigator to the data.py, use print(f'index={index}') to observe the situation:

Then I run train_test.py again. Below I pasted the full display of the error report. I found that when running epoch0, the index suddenly became very big. This is what caused the error. I don't know how to solve it.

Validation sanity check: 0%| | 0/1 [00:00<?, ?it/s]index = 0
index = 1
index = 2
index = 3
index = 4
index = 5
index = 6
index = 7
index = 8
index = 9
index = 10
index = 11
index = 12
index = 13
index = 14
index = 15
index = 16
index = 17
index = 18
index = 19
index = 20
index = 21
index = 22
index = 23
index = 24
index = 25
index = 26
index = 27
index = 28
index = 29
index = 30
index = 31
Global seed set to 42
Epoch 0: 0%| | 0/9 [00:00<?, ?it/s]index = 1088
Traceback (most recent call last):
File "train_test.py", line 183, in
main(args)
File "train_test.py", line 135, in main
trainer.fit(runner, datamodule=pipeline)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 463, in fit
self._run(model)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 761, in _run
self.dispatch()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 802, in dispatch
self.accelerator.start_training(self)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 96, in start_training
self.training_type_plugin.start_training(trainer)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 144, in start_training
self._results = trainer.run_stage()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 812, in run_stage
return self.run_train()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 874, in run_train
self.train_loop.run_training_epoch()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\training_loop.py", line 491, in run_training_epoch
for batch_idx, (batch, is_last_batch) in train_dataloader:
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\profiler\profilers.py", line 112, in profile_iterable
value = next(iterator)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\supporters.py", line 530, in prefetch_iterator
last = next(it)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\supporters.py", line 464, in next
return self.request_next_batch(self.loader_iters)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\trainer\supporters.py", line 478, in request_next_batch
return apply_to_collection(loader_iters, Iterator, next)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pytorch_lightning\utilities\apply_func.py", line 85, in apply_to_collection
return function(data, *args, **kwargs)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data\dataloader.py", line 530, in next
data = self._next_data()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data\dataloader.py", line 1224, in _next_data
return self._process_data(data)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data\dataloader.py", line 1250, in _process_data
data.reraise()
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch_utils.py", line 457, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data_utils\worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\24061\Desktop\MERcode\EMOPIA_cls-main\emopia_retrain\audio_cls\src\data.py", line 34, in getitem
audio_fname = self.fl.iloc[index].name
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pandas\core\indexing.py", line 1767, in getitem
return self._getitem_axis(maybe_callable, axis=axis)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pandas\core\indexing.py", line 2137, in _getitem_axis
self._validate_integer(key, axis)
File "C:\Users\24061\anaconda3\envs\emopia\lib\site-packages\pandas\core\indexing.py", line 2062, in _validate_integer
raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds

index = 9415
index = 9102
wandbindex = 6822
: Waiting for W&B process to finish... (failed 1). Press Ctrl-C to abort syncing.
index = 9948
index = 2649
index = 698101 MB of 0.001 MB uploaded (0.000 MB deduped)
index = 3089
wandb:
wandb: Synced playful-thunder-62: https://wandb.ai/yiliazhu/uncategorized/runs/wiq17y5e
wandb: Synced 6 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: .\wandb\run-20220531_235944-wiq17y5e\logs

Hi,
I tried the midi_cls/train_test.py on environment below:

pytorch == 1.8.0
python == 3.8.5
CUDA ==10.1

I trained the models with A/V/AV best hyperparameters that provided in best weights folder.
But got different results from the json file provided in best weight file.

Below is the hyperparams and results log produced by original train_test.py code. (I transformed them to .txt for uploading.)
arva_hparams.txt
arva_results_last.txt
av_hparams.txt
av_results_last.txt

• I already added global seed setting in the script.

I'm really confused about the reason, is there any way to make sure what's going wrong?
thanks.

Link to data files in 'Training from scratch' area got 404 failed.
https://github.com/SeungHeonDoh/EMOPIA_cls/blob/main

Hi,
I want to test the model in my data and I'm using CPU. but I get the error ''Torch not compiled with CUDA enabled''.
Could somebody help me what the problem is?

As below, the matlab features are not included in the repo.

Is there any way to download it?
thanks!

I tested inference_batch.py on dataset/split/test.csv and got 0.57,0.744,0.744 on AV/A/V separately.
The models I used were download from https://drive.google.com/u/0/uc?id=1L_NOVKCElwcYUEAKp1-FZj_G6Hcq2g2c&export=download (which were provided in README.md).