gb96 / gruv Goto Github PK

Dropout between model layers is a regularization technique for reducing overfitting.

Allow two distinct dropout parameters dropout_W and dropout_U:

• dropout_W: float between 0 and 1. Fraction of the input units to drop for input gates.
• dropout_U: float between 0 and 1. Fraction of the input units to drop for recurrent connections.

See Keras GRU and LSTM documentation.

Parameters to be specified in config.nn_config and used when creating the model.

Some tests are better than none!

Currently the model is created in both train and generate

Consequently whenever a change is made to one creation call, it needs to be manually replicated in the other file. Much better if there was a common function in network_utils.create_model()

The model creation functions defined in nn_utils.network_utils include an optional parameter num_recurrent_units=1

None of the calls to these functions from train.py and generate.py specify a value for this parameter, so by default there is only ever 1 hidden layer.

Add a parameter to config.nn_config for the number of hidden layers and then use the value of that parameter wherever the model is created.