Speech Recognition with BVLC caffe

Speech Recognition with the caffe deep learning framework

Caution: This project is very fresh and only the first of three milestones is accomplished: Even now it might be useful if you just want to train a handful of commands/options (1,2,3..yes/no/cancel/...)

1. training spoken numbers:

• get spectogram training images from http://pannous.net/spoken_numbers.tar (470 MB)
• start ./train.sh
• test with ipython notebook test-speech-recognition.ipynb or caffe test ... or <caffe-root>/python/classify.py
• 99% accuracy, nice!
• online recognition and learning with ./recognition-server.py and ./record.py scripts

2. training words:

• 4GB of training data *
• net topology: work in progress ...
• todo: use new caffe LSTM layers etc
• todo: add extra categories for a) silence b) common noises like typing, achoo c) ALL other noises

3. training speech:

• todo!
• 100GB of training data here: http://www.openslr.org/12/
• TIMIT dataset $27,000.00 membership fee?
• combine with google n-grams

Theoretical background: papers

A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In ICML, 2014

O. Vinyals, S. V. Ravuri, and D. Povey. Revisiting recurrent neural networks for robust ASR. In ICASSP, 2012

Ng et al / Baidu

Hinton et al / Toronto

good old Hinton

Also see the Kaldi project, which seems a bit messy but already uses deep learning with LSTM