Extend the Module class about singa HOT 7 CLOSED

@XJDKC In this case, I suggest we can use the buffer operation feature so there won't be any actual run.
Then, in case if we don't want to use graph after getting the parameter size, we can use ResetGraph to clear the buffer, then turn off buffer and run without graph.

from singa.

I worry that if we move the param creation and initialization into the forward propagation stage, those initialization operations will also be buffered and included in the graph.

from singa.

I worry that if we move the param creation and initialization into the forward propagation stage, those initialization operations will also be buffered and included in the graph.

@XJDKC Can use ResetGraph to reset the buffer?

from singa.

I worry that if we move the param creation and initialization into the forward propagation stage, those initialization operations will also be buffered and included in the graph.

@XJDKC Can use ResetGraph to reset the buffer?

I prefer adding an initialization function for Layer, the module just needs to call the init function of each layer to initialize the whole model.

from singa.

I worry that if we move the param creation and initialization into the forward propagation stage, those initialization operations will also be buffered and included in the graph.

@XJDKC Can use ResetGraph to reset the buffer?

I prefer adding an initialization function for Layer, the module just needs to call the init function of each layer to initialize the whole model.

@XJDKC
Yes, in the following example the parameter of the layer is initialized in the call function, where its shape is infered from the input size of that layer

from singa.

I worry that if we move the param creation and initialization into the forward propagation stage, those initialization operations will also be buffered and included in the graph.

@XJDKC Can use ResetGraph to reset the buffer?

I prefer adding an initialization function for Layer, the module just needs to call the init function of each layer to initialize the whole model.

@XJDKC
Yes, in the following example the parameter of the layer is initialized in the call function, where its shape is infered from the input size of that layer

singa/python/singa/autograd.py

Line 3406 in 19f2020

self.W = Tensor(shape=(self.handle.weights_size,),

Can this case be handled?

The handle can be created in the init function, and then W can be initialized by the handle.

from singa.

#696

from singa.