buddy's Introduction

BUDDy: Single-channel Blind Unsupervised Dereverberation with Diffusion Models

This is the official code for our paper BUDDy: Single-channel Blind Unsupervised Dereverberation with Diffusion Models

We invite you to check our companion website for listening samples and insights into the paper

1 - Requirements

Install required Python packages with python -m pip install -r requirements.txt

2 - Checkpoints

You can access our pretrained checkpoint, trained on VCTK anechoic speech, at the following link

3 - Testing

You can launch blind dereverberation with bash test_blind_dereverberation.sh. You can launch informed dereverberation with bash test_informed_dereverberation.sh. In both cases, do not forget to add the path to the pretrained model checkpoint in the bash file (i.e. replace ckpt=<pretrained-vctk-checkpoint.pt> with your path). The directory tree in audio_examples/ contains an example test set to reproduce the results.

4 - Training

You can retrain an unsupervised diffusion model on your own dataset with bash train.sh. Do not forget to fill in the path to your training and testing dataset (i.e. replace dset.train.path=/your/path/to/anechoic/training/set and so on)

5 - Citing

If you used this repo for your own work, do not forget to cite us:

@bibtex

@article{moliner2024buddy,
    title={{BUDD}y: Single-channel Blind Unsupervised Dereverberation with Diffusion Models},
    author={Moliner, Eloi and Lemercier, Jean-Marie and Welker, Simon and Gerkmann, Timo and V\"alim\"aki, Vesa},
    year={2024},
    journal={arXiv 2405.04272}
}

buddy's People

Contributors

Stargazers

Watchers

buddy's Issues

A strange problem

Hi,
thank you for your contribution. I test these code but occur a strange problem:

  File "test.py", line 103, in main
    _main(args)
  File "test.py", line 98, in _main
    tester.do_test()
  File "/home/wangyutian/works/buddy-master/testing/tester.py", line 233, in do_test
    self.test_dereverberation(m, blind=True)
  File "/home/wangyutian/works/buddy-master/testing/tester.py", line 153, in test_dereverberation
    pred = self.sampler.predict_conditional(y, operator_blind if blind else operator_ref, shape=(1,seg.shape[-1]), blind=blind)
  File "/home/wangyutian/works/buddy-master/testing/EulerHeunSamplerDPS.py", line 204, in predict_conditional
    return self.predict(shape, y.device, blind)
  File "/home/wangyutian/works/buddy-master/testing/EulerHeunSamplerDPS.py", line 176, in predict
    x, x_den = self.step(x, t[i] , t[i+1], gamma[i], blind)
  File "/home/wangyutian/works/buddy-master/testing/EulerHeunSamplerDPS.py", line 119, in step
    x_den = self.get_Tweedie_estimate(x_hat, t_hat)
  File "/home/wangyutian/works/buddy-master/testing/Sampler.py", line 71, in get_Tweedie_estimate
    x_hat = self.diff_params.denoiser(x.unsqueeze(1), self.model, t_i).squeeze(1)
  File "/home/wangyutian/works/buddy-master/diff_params/shared.py", line 120, in denoiser
    return cskip * xn + cout * net(cin * xn, cnoise)  #this will crash because of broadcasting problems, debug later!
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/wangyutian/works/buddy-master/networks/ncsnpp.py", line 503, in forward
    x_spec=super().forward(x_spec, time_cond=time_cond)
  File "/home/wangyutian/works/buddy-master/networks/ncsnpp.py", line 293, in forward
    x_chans.append(torch.cat([
  File "/home/wangyutian/works/buddy-master/networks/ncsnpp.py", line 294, in <listcomp>
    torch.cat([x[:,[chan+in_chan],:,:].real, x[:,[chan+in_chan],:,:].imag ], dim=1) for in_chan in range(self.input_channels // 2)],
RuntimeError: index does not support automatic differentiation for outputs with complex dtype.

is there any solution? thank you !

Recommend Projects

sp-uhh / buddy Goto Github PK

buddy's Introduction

BUDDy: Single-channel Blind Unsupervised Dereverberation with Diffusion Models

1 - Requirements

2 - Checkpoints

3 - Testing

4 - Training

5 - Citing

buddy's People

Contributors

Stargazers

Watchers

Forkers

buddy's Issues

A strange problem

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent