Implement NengoDL version of ModelCheckpoint about nengo-dl HOT 4 CLOSED

If you use sim.save_params/load_params then this will work. When using keras_model.save_weights you're also saving the internal simulation state (which has a minibatch dimension), which is why those parameters don't transfer between models with different minibatch size.

from nengo-dl.

Good to know. For reference the reason [save|load]_weights is being used as opposed to [save|load]_params is because I'm using the ModelCheckpoint callback with save_weights_only=True. Setting this to False triggers some issue in the serialization/deserialization logic when trying to load them back in.

from nengo-dl.

Going to use this issue to track the idea of implementing some thin wrapper around ModelCheckpoint that will call sim.save_params instead of sim.keras_model.save_weights.

from nengo-dl.

Note to future self. An easier fix might be to store simulator state as a simple tf.Variable (not added through layer.add_weights). Then Keras wouldn't track it, and sim.keras_model.save_weights would produce the same behaviour as sim.save_params (only saving the trainable parameters of the model).

from nengo-dl.