Moving Network objects to different device? about bindsnet HOT 11 CLOSED

@sharath @djsaunde Has there been any movement on this? If not it is something I can crank out rev 1 of this week. I will probably first look into subclassing torch.nn.Module as it would provide the most functionality for the least work. Some benefits to exploit out of the box:

1. .to(device) will allow for management of devices directly
2. saving of Networks and torch.nn.Modules together (e.g. a reservoir type setup as in examples/mnist/minimal_reservoir.py)
3. this enable the use of the full pytorch data loading pipeline with pin_memory enabled which provides asynchronous transfer to the gpu

from bindsnet.

from bindsnet.

@djsaunde so I've put some initial work into this. I need to check why the tests are failing on the build server. However, this should give you an initial look into how it could work. Almost everything utilizes the buffer functionality in torch.nn.Module, except for the weights and biases which are Parameters without gradients. I haven't migrated the Network to inherit from torch.nn.Module yet as I'm still thinking about how the Monitors will play out.

from bindsnet.

register_buffer just registers a tensor to be tracked by pytorch to help with to() and state_dict(), so there's no real performance hit here.

The only reason would be to keep __init__.py as strictly management of the submodule, which is traditionally what I've kept __init__.py for. This would be more philosophy of design for bindsnet than anything else.

I think this would be a good feature, something I've been thinking about as I've been writing more code as well. But is definitely out of scope for this.

from bindsnet.

Hey @sharath, any movement on this feature?

from bindsnet.

@djsaunde I'll work on this soon.

from bindsnet.

@k-chaney it'd be great if you could take a look at this. I think the preferred approach is having Nodes, Connections, and Network objects subclass torch.nn.Module, like you said.

from bindsnet.

I think @sharath will be occupied this week.
Feel free to start on it.

from bindsnet.

Great! The code changes look good so far. Do you think it will have any impact on runtime performance? What exactly does register_buffer do?

One thing I'm a little wary of is migrating the Network code to bindsnet.network.network; that seems a little redundant, and it might be better to keep it in the __init__.py file. Unless there's a good reason not to?

It may be interesting to optionally keep gradients for "smoothed" versions of nodes / topology objects (i.e., without discontinuities), but that will probably be better off as future work.

from bindsnet.

@djsaunde what's the best way to look at the travis log? All of the tests pass for me locally. (nevermind I found the link to the travis-ci website)

from bindsnet.

This has been initially completed in #266

from bindsnet.