Can not get reproducible training of modules about hub HOT 3 CLOSED

Thank you, Jonas, for this insightful report.

Indeed, a module_fn passed to hub.create_module_spec() executes in its own tf.Graph whose seed is not set. This may happen outside an Estimator's model_fn, so there is no general way to get hold of the eventual value of RunConfig.tf_random_seed.

Perhaps the best way to do this in general is to call tf.set_random_seed(seed) in the module_fn like your workaround suggests, with a seed that is not None? (I suspect you might be getting None in the line before, from an otherwise untouched Graph.)

from hub.

Thanks for the explanation. Any odds that in the future, the estimator class could overwrite the modules graph seed when training start, e.g. when the module exporter is registered to the estimator?

from hub.

Module export takes the weights from after training, but the graph(s) of the module are unchanged from when hub.create_module_spec() was called on the module_fn, and that may happen far away from the Estimator that instantiates that module spec into a module eventually. So, no, with the current design, I don't see a better way than calling tf.set_random_seed() in the module_fn.

from hub.