Two models with tied inputs about fold HOT 2 CLOSED

Hi thanks for asking. The key thing to realize is that variables do not get created when you construct a TF rnn cell, they get created when you invoke the cell's call operator. When you use an RNN cell or other TF layer with Fold, you need to delegate this call to the Fold library (in order to wire up the corresponding subgraph for dynamic batching).

Rather than managing this yourself, we recommend using the scoped layer helper class. The simple rule to follow here is to put every rnn cell you want to use with Fold inside of a separate scoped layer, like we do with tree_lstm in the example notebook. When you do this, every rnn cell will have its own separate scope and hence variables regardless of how many times you use it in a Fold model, with no need for explicit variable scoping.

So for your particular use-case, I'd recommend making tree_lstm an argument to logits_and_state() rather than a global variable. Then at the top level you can create two separate lstm cells in separate scoped layers, and explicitly specify which one you want to use when logits_and_state gets called. You might also want to have separate output layers as well, in which case you'll want to add the output layer as an argument to logits_and_state as well. Do note that because td.FC is a built-in fold layer rather than a TF builtin, you don't need to bother wrapping it in a scope layer, we take care of that for all Fold layers.

from fold.

P.S. One more thing, I noticed that your root block (what you use to create the compiler) is (model1, model2). This is implicitly creating a record block, which is used for destructuring inputs. E.g. in this case the overall model will expect a pair of inputs and will invoke model1 with input[0] and model2 with input[1]; probably not what you want!

If instead you say td.AllOf(model1, model2), both models will get input, and the output will be a pair consisting of the two models respective outputs.

from fold.