Comments (4)
We use the standard activation function for (BI)LSTM. You can find more details by looking at Section 2 of the following paper
http://www.cs.cmu.edu/~ymiao/pub/is2015_lstm.pdf
The cell and hidden output dimension is the same. It can be set by changing "lstm_cell_dim" in run_ctc_xxx.sh. By default it's set to 320
from eesen.
Thanks a lot!
Can I insert a sub-sampling layer between two BILSTM layers in eesen? The sub-sampling layer is expected to play reducing net parameters like SVD.
Can the different cell and hidden output dimension be supported in eesen?
from eesen.
When you said "sub-sampling", I assume that you meant sub-sampling the frames. This is hard because we pad the utterances to the same length within a batch. Your sub-sampling layer needs to figure out which of the frames are not padding frames and does the backpropagation accordingly.
You may simply try inserting an affine transform between two LSTM layers, which projects LSTM outputs into a presumably lower-dimensional vector. But this may introduce training instability as the projection layer is not recurrent.
In its vanilla definition, LSTM has to have the same cell and output dimension. Google's LSTM with recurrent projection layer is not supported.
from eesen.
Thanks!
from eesen.
Related Issues (20)
- Clean up v2 for swb
- DeepBiLSTM HOT 2
- Missing label.counts HOT 3
- Query on LibriSpeech Character Error Rate HOT 2
- difference in output labels HOT 1
- Memory Leak HOT 1
- failed: Dim() == v.Dim() HOT 2
- Potential overflow when calculating exp
- Clarification Regarding Using WFST decoding HOT 1
- Installing error HOT 8
- Training Error when run tedlium recipe HOT 2
- LatticeFasterDecoder failed with "link_extra_cost == link_extra_cost" HOT 1
- Cannot install openfst-1.4.1 HOT 2
- Read failure in ReadBasicType, file position is -1, next char is -1
- KALDI_ASSERT: at train-ctc-parallel:AddMatMat:cuda-matrix.cc:570, failed: m == NumCols()
- Why do we need space and unk symbols in the char mode for acoustic model? HOT 6
- Why do we need to compile the tokens to FST in wsj recipe?
- Can not run training program with cuda 10.2 HOT 3
- Librispeech - Training starting error HOT 3
- Determinizability of TLG.fst in the phonetic case
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eesen.