mem_cell_ct =100 but only 4 LSTM nodes about lstm HOT 5 OPEN

As far as I can see, he choose for 100 hidden cells because that is the double of the randomly chosen 50 input cells.

Choosing the amount of hidden cells could be a bit of a guessing game. Trying different numbers to see what works best. This is the case for most neural networks (as far as I know of, all of them)

There are tricks that makes the guessing simpler. Like, making the hidden layer use twice as much cells as input cells (as in this example). Or choosing the average of the input + output cells. I for example tested it with 26 ((50inputs + 1output) / 2) hidden cells. And that resulted in about the same loss.

But there are also networks that try to find the amount of hidden cells for them self. More explanation can be found in this thread: http://stackoverflow.com/questions/3345079/estimating-the-number-of-neurons-and-number-of-layers-of-an-artificial-neural-ne

About LSTM nodes. They seem to me that their only use is to remember the input at a certain time step and couple that with the states of all cells of the LstmState object. This way the back propagation could work. (you need the input values times the difference to find the error of the weights). Summary in the program: LstmNode is used to couple self.state (all the values in all the cells) with self.xc (the input).

from lstm.

In fact, the project only uses one memory cell, which has big difference from LSTM node. The memory cell in this work really means the hidden size of the cell.

from lstm.

from lstm.

I think the notation in the literature is not consistent. I refer to https://arxiv.org/abs/1506.00019

from lstm.

@zackchase As far I understood, the terminology can be confusing.
However, what @ScottMackay2 writes makes sense to me from the code and implementation point of view of unrolling the LSTM network.

In lstm.py, LSTM nodes refer to the repeating (same weight parameters, varying cell states) hidden layer of memory cells in the unrolled network. In example_0, the number of nodes is set programmatically to the maximum number (n=4) of time steps of the random 50 dimensional input sequence.

from lstm.