Light

sungwon23 / bsrnn Goto Github PK

View Code? Open in Web Editor NEW

67.0 2.0 12.0 61.4 MB

Python 100.00%

bsrnn's Introduction

BSRNN

Unofficial PyTorch implementation of the paper "HIGH FIDELITY SPEECH ENHANCEMENT WITH BAND-SPLIT RNN" (https://arxiv.org/abs/2212.00406) on VCTK-DEMAND Dataset (https://datashare.ed.ac.uk/handle/10283/2791).

Result

Choosed parameter settings

N (feature dimension) : 64, L (the number of lstm layers) : 5

	PESQ	SSNR	STOI
Noisy	1.97	1.68	0.91
BSRNN(N=64, L=5)	3.10	9.56	0.95

Audio files are in saved_tracks_best folder.

Train and inference

1. Dependencies:

Used packages are can be installed by:

pip install -r requirements.txt

2. Download dataset:

Download VCTK-DEMAND dataset (https://datashare.ed.ac.uk/handle/10283/2791), change the dataset dir:

-VCTK-DEMAND/
  -train/
    -noisy/
    -clean/
  -test/
    -noisy/
    -clean/

3. Train:

python train.py --data_dir <dir to VCTK-DEMAND dataset>

If you want to adjust parameters (N, L) of the model, change the value of variables in train.py.

self.model = BSRNN(num_channel=64, num_layer=5).cuda()

4. Inference and metric evaluation:

python evaluation.py --test_dir <dir to VCTK-DEMAND/test> --model_path <path to the best ckpt>

Reference

https://github.com/ruizhecao96/CMGAN (MIT License)

bsrnn's People

Contributors

Stargazers

Watchers

Forkers

sherryyu33 shaun95 ryanzlay wendongj katori-ouo comeonlby runngezhang kingfener tungyuyoung yoshikimas xxmiprai meadow163

bsrnn's Issues

Questions about code reproduction effects

First of all, thank you for your work.
I would like to ask if the results obtained by that code can get the results described in the paper and if they can beat MTFAA and FRCRN in terms of objective scores?

Real-time mode

Hi, does your code have real-time mode? What I mean is there possibility in your code for batch audio processing?

model issue

Can this model be used for speaker separation, and how can it be done?

reproductioin

Hello，thank you for your sharing and generous！
But I got a problem when I reproduct your result with totally the same code. I use VoiceBank+Demand dataset , sampling rate 16kHz. And I split the 28 speakers in the trainset into train and test for 26 and 2 spkears ,respectively. I set batch size as 18, and train epoch 100 in total.
When I test the model in VoiceBank+Demand testset, BUT I got PESQ at 1.97, STOI 0.92, it's weird and just like the model doesn't has been trained. Are you use the same testset in train and test? Or can you share your opinion about this question?
Thanks a lot.

lstm t/k order

I think the lstm t/k sequence should be lstm_t_i lstm_k_i lstm_t_i+1 lstm_k_i+1 ?

https://github.com/sungwon23/BSRNN/blob/main/module.py#L46-L60

view_as_real is only supported for complex tensors

Hello! I tried to change the code to dp training form and found this problem
in: module.py line40. I printed the input tensor for model and found that the data form of dp multi-GPU is different from the original single GPU

Causal system

Hello! I want to confirm to you that is this code already a causal system? If not, which part leads to non-causal processing? Thanks.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.