jayanthkoushik / sgd-feedback Goto Github PK

In eve.py, line 32:

ch_fact_lbound = K.switch(K.greater(loss, loss_prev), 1+self.thl, 1/(1+self.thu))
ch_fact_ubound = K.switch(K.greater(loss, loss_prev), 1+self.thu, 1/(1+self.thl))

This implies that if loss > loss_prev, then ch_fact_lbound = 1 + self.thk

If I'm not mistaken, this is the opposite of what's written in the paper.

I ran into this issue when I tried to implement the algorithm independently. If you replace K.greater by K.lesser (following what's written in the paper) then the loss quickly diverges.

For what it's worth, I have a somewhat simpler keras implementation of Eve here: https://github.com/tdeboissiere/DeepLearningImplementations/blob/master/Eve/Eve.py