from example in doc (my brain bug or tff bug ?) about fold HOT 7 CLOSED

you need to set state_is_tuple to be false, so the cell and hidden state are concatenated together.

i.e

# Create RNN cells using the TensorFlow RNN library
char_cell = td.ScopedLayer(
    tf.contrib.rnn.BasicLSTMCell(num_units=16, state_is_tuple=False),
    'char_cell'
)
word_cell = td.ScopedLayer(
    tf.contrib.rnn.BasicLSTMCell(num_units=32, state_is_tuple=False),
    'word_cell'
)

The example should work. I'll leave it up to you to figure out what the output of the eval actually represents :)

from fold.

from fold.

According to section 3.3 of hierarchical LSTMs, the word_lstm gets the hidden outputs of char_lstm as inputs. So, I think the correct way to construct a word_lstm cell with char_cell's state_is_tuple=True is here below:

import tensorflow as tf
import tensorflow_fold as td

# Create RNN cells using the TensorFlow RNN library
char_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=16), 'char_cell')
word_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=32), 'word_cell')

# character LSTM converts a string to a word vector
char_lstm = (td.InputTransform(lambda s: [ord(c) for c in s]) >>
             td.Map(td.Scalar('int32') >>
                    td.Function(td.Embedding(128, 8))) >>
             td.RNN(char_cell))
# word LSTM converts a sequence of word vectors to a sentence vector.
word_lstm = td.Map(char_lstm >> td.GetItem(1) >> td.GetItem(1)) >> td.RNN(word_cell)

sess = tf.InteractiveSession()

word_lstm.eval(["bon","ben"])

I add one more >> td.GetItem(1) behind >> td.GetItem(1), so that word_lstm can get the hidden_output of one char_lstm instead of (cell_status, hidden_output) of one.

Additionally, the content of test_hierarchical_rnn in blocks_test.py also need to be revised.

Wish this is helpful!

from fold.

Will Try :)

but are you sure ? if i remember the char_lstm_output[1] is just 32 float long...
anyway i will check !

Yeah you right but with state_is_tuple=True ! I'm dumb 8~|
:)

from fold.

@benjaminderei
No. char_lstm_output[1] is an instance of LSTMStateTuple if you set state_is_tuple=True; otherwise, it is 32 float numpy array.

from fold.

I have edited my previous post :)

from fold.

@benjaminderei glad to help you.

from fold.