The deep-encoder-decoder-conv-tasnet from jusperlee

deep-encoder-decoder-conv-tasnet's Introduction

Hey 👋🏽, I'm Kai Li!

My name is Kai Li (Chinese name: 李凯). I'm a second-year master student at Department of Computer Science and Technology, Tsinghua University, supervised by Prof. Xiaolin Hu (胡晓林). I am also a member of TSAIL Group directed by Prof. Bo Zhang (张拨) and Prof. Jun zhu (朱军). I am an intern at Tencent AI Lab, mainly doing research on causal speech separation, supervised by Yi Luo (罗艺).

🤗 These works are open source to the best of my ability.

🤗 I am currently doing research on multimodal speech separation, and am interested in other speech tasks (e.g., pre-training models and neuralscience). If you would like to collaborate, please contact me. Many thanks.

🔖 Homepages

: Kai Li : Jusper Lee : cslikai.cn

📅 News

2023.07: 🎲 One paper is accepted by ECAI 2023.
2023.05: 🧩 Two papers are accepted by Interspeech 2023.
2023.05: 🎉 We won the first prize 🥇 of the Cinematic Sound Demixing Track 23 in the Leaderboard A and B.
2023.05: 🎉 We won the first prize 🥇 of the ASC23 and Best Application Award.
2023.04: 🎲 One paper is appeared by Arxiv.
2023.02: 🧩 One paper is accepted by ICASSP 2023.
2023.01: 🧩 One paper is accepted by ICLR 2023.

📰 Selected Publications:

See Google Scholar for a full list of publications.

Speech Separation

An efficient encoder-decoder architecture with top-down attention for speech separation. Kai Li, Runxuan Yang, Xiaolin Hu. ICLR 2023.
An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits. Kai Li, Fenghua Xie, Hang Chen, Kexin Yuan, Xiaolin Hu. Arxiv 2022.
Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network Xiaolin Hu, Kai Li, Weiyi Zhang, Yi Luo, Jean-Marie Lemercier, Timo Gerkmann. NeurIPS 2021.

Neuroscience

Inferring mechanisms of auditory attentional modulation with deep neural networks. Ting-Yu Kuo, Yuanda Liao, Kai Li, Bo Hong, Xiaolin Hu. Neural Computation 2022.

Cloud Removal

PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-Performance Cloud Removal from Multi-temporal Satellite Imagery. Xuechao Zou, Kai Li, Junliang Xing, Pin Tao#, Yachao Cui. ECAI 2023.

Super Resolution

A Survey of Single Image Super Resolution Reconstruction. Kai Li, Shenghao Yang, Runting Dong, Jianqiang Huang, Xiaoying Wang. IET Image Processing 2020.
Single Image Super-resolution Reconstruction of Enhanced Loss Function with Multi-GPU Training. Jianqiang Huang, Kai Li, Xiaoying Wang. ISPA 2019.

deep-encoder-decoder-conv-tasnet's People

Contributors

Stargazers

Watchers

deep-encoder-decoder-conv-tasnet's Issues

Error message on Asteroid - list output

Hi! Nice repo, @JusperLee :)
I am trying to use Conv-Tasnet-Deep-w-dilation.py with Asteroid but I get the error below when training. Edit: Solved this by returning torch.unsqueeze(s[0], dim=1) for when num_spks=1.

  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 90, in launch
    return function(*args, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 645, in _fit_impl
    self._run(model, ckpt_path=self.ckpt_path)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1098, in _run
    results = self._run_stage()
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1177, in _run_stage
    self._run_train()
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1190, in _run_train
    self._run_sanity_check()
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1262, in _run_sanity_check
    val_loop.run()
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 152, in advance
    dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 137, in advance
    output = self._evaluation_step(**kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 234, in _evaluation_step
    output = self.trainer._call_strategy_hook(hook_name, *kwargs.values())
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1480, in _call_strategy_hook
    output = fn(*args, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/strategies/ddp.py", line 360, in validation_step
    return self.model(*args, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1040, in forward
    output = self._run_ddp_forward(*inputs, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1000, in _run_ddp_forward
    return module_to_run(*inputs[0], **kwargs[0])
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/pytorch_lightning/overrides/base.py", line 110, in forward
    return self._forward_module.validation_step(*inputs, **kwargs)
  File "/media/jvel/data/repos/asteroid/asteroid/engine/system.py", line 130, in validation_step
    loss = self.common_step(batch, batch_nb, train=False)
  File "/media/jvel/data/repos/asteroid/asteroid/engine/system.py", line 102, in common_step
    loss = self.loss_func(est_targets, targets)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/media/jvel/data/repos/asteroid/asteroid/losses/pit_wrapper.py", line 100, in forward
    pw_losses = self.loss_func(est_targets, targets, **kwargs)
  File "/home/jvel/anaconda3/envs/asteroid/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/media/jvel/data/repos/asteroid/asteroid/losses/sdr.py", line 46, in forward
    if targets.size() != est_targets.size() or targets.ndim != 3:
AttributeError: 'list' object has no attribute 'size'