Can CNN buffers be shared ? about opennmt HOT 2 CLOSED

gradInput tensors can always be shared unless they are exposed outside the network but the code handles that for you.

output tensors can be shared if they are not used in the backward pass. You should take a look at the implementation of every new modules you introduce in the code and apply this rule:

1. If the content of input is used in updateGradInput or accGradParameters, add the module in the protectInput table.
2. If the content of self.output is used in updateGradInput or accGradParameters, add the module in the protectOutput table.

For example, see the modules that are already blacklisted in MemoryOptimizer.lua and how they use their input or output in the backward pass.

It forces you to know in details the modules you use but the optimization is efficient. To validate your approach, you should get the same perplexity with and without memory optimization.

from opennmt.

@guillaumekln Thank you so much!

from opennmt.