About net topology and activation function in eesen about eesen HOT 4 CLOSED

We use the standard activation function for (BI)LSTM. You can find more details by looking at Section 2 of the following paper
http://www.cs.cmu.edu/~ymiao/pub/is2015_lstm.pdf

The cell and hidden output dimension is the same. It can be set by changing "lstm_cell_dim" in run_ctc_xxx.sh. By default it's set to 320

from eesen.

Thanks a lot!
Can I insert a sub-sampling layer between two BILSTM layers in eesen? The sub-sampling layer is expected to play reducing net parameters like SVD.
Can the different cell and hidden output dimension be supported in eesen?

from eesen.

When you said "sub-sampling", I assume that you meant sub-sampling the frames. This is hard because we pad the utterances to the same length within a batch. Your sub-sampling layer needs to figure out which of the frames are not padding frames and does the backpropagation accordingly.

You may simply try inserting an affine transform between two LSTM layers, which projects LSTM outputs into a presumably lower-dimensional vector. But this may introduce training instability as the projection layer is not recurrent.

In its vanilla definition, LSTM has to have the same cell and output dimension. Google's LSTM with recurrent projection layer is not supported.

from eesen.

Thanks!

from eesen.