y_list dimension about lstm HOT 7 CLOSED

@evbo ,
you are correct, my example was intended as multiple inputs, single output. Typically, there is an inner product layer + softmax on top of the network which takes the hidden state at every time point and predicts something (e.g. the current character). Since this would involve writing another layer and I wanted to keep the code as simple as possible, I implicitly made this extra layer w^T h, where w = [1,0,...] and h is the value of the hidden layer. Would it be less confusing if I added another layer?

With a single input, single ouput, the network simply does not have enough complexity to predict a complex sequence. This would work a lot better if you added an inner product layer on top in order to decode the internal dynamics.

from lstm.

Thanks for confirming. This makes sense. I think stylistically is where people get confused - some come from a background where it is customary to have the output squashing function and some do not. It seems, even according to Lipton's paper, that the extra squashing function is optional with no evidence of it being necessary.

from lstm.

kindly help me whats going over there
y_list = [-0.5,0.2,0.1, -0.5]
input_val_arr = [np.random.random(x_dim) for _ in y_list]

for _ in y_list
what does this ( _ ) means in above for loop statement

and also please elaborate me i am new in python and nn

from lstm.

You could search the _ syntax for yourself on google (it is just a convention for saying that that variable will not be used). And this was clearly not a Python tutorial. You should follow one first.

from lstm.

from lstm.

Sorry. Was not my intention to bring this the wrong way.

from lstm.

@evbo ,
you are correct, my example was intended as multiple inputs, single output. Typically, there is an inner product layer + softmax on top of the network which takes the hidden state at every time point and predicts something (e.g. the current character). Since this would involve writing another layer and I wanted to keep the code as simple as possible, I implicitly made this extra layer w^T h, where w = [1,0,...] and h is the value of the hidden layer. Would it be less confusing if I added another layer?

With a single input, single ouput, the network simply does not have enough complexity to predict a complex sequence. This would work a lot better if you added an inner product layer on top in order to decode the internal dynamics.

Sincerely hope you can answer my question ASAP，its killing me. From your code, it looks like a multiple-input multiple-output LSTM. You took the first value of state.h in the four Lstm_time node as the final output. Isn't this multiple output? As far as i know, single output means only take the final node'output as result, am i right?

from lstm.