mcmachado / count_based_exploration_sr Goto Github PK

Hi, just checking for my own sanity: in both tabular/sarsa_agent.py and tabular/sarsa_sr_agent.py you use the following to update q:

def update_q_values(self, s, a, r, next_s, next_a):
self.q[s][a] = self.q[s][a] + self.alpha * (r + self.gamma - (1.0 - self.env.is_terminal()) *
self.q[next_s][next_a] - self.q[s][a])

why is it r + self.gamma - (1.0-terminal)q[next_s][next_a] ... rather than r + self.gamma(1.0-terminal)*q[next_s][next_a] ?
you do the same with the SR update. This is a mistake, right? (I can see it might not effect the results too much, but I just wanted to check!)

Hi,

I noticed that the code is not compatible with the last tensorflow and numpy versions. After some inspection, I found out that it runs properly with versions 1.8.0 and 1.14.0 respectively. Hope it helps :)

Cheers,

Daniel