hannw / sgrnn Goto Github PK

View Code? Open in Web Editor NEW

40.0 2.0 9.0 4.25 MB

Tensorflow implementation of Synthetic Gradient for RNN (LSTM)

Home Page: https://hannw.github.io/posts/synthetic-gradient-rnn

Python 100.00%

recurrent-neural-networks synthetic-gradients tensorflow rnn dni decoupled-neural-interfaces metalearning lstm

sgrnn's People

Contributors

Stargazers

Watchers

Forkers

kastnerkyle shubhampachori12110095 dbverstr5324 jdc08161063 wanjinchang sufeidechabei ychen404 guillaumebellec robinkumarsharma03

sgrnn's Issues

Difference between sg_target and gradient in the implementation

Can I know if possible the difference between sg_target and gradient in your code. Mainly which one back propagate the synthetic gradient all the way back? Is it the future_grad?

I know this is not an issue but I would like to re-write the code on my own in tensorflow.

Any help is much appreciated!!

Training diverges

I ran the README.md example and training seems to consistently diverge. I'm sure I've got something wrong but before I start debugging, have you managed to reproduce the PTB results of the paper with your implementation?

Recommend Projects