Not sample efficient enough about async-rl HOT 4 OPEN

Following the authors' feedback, now it's only slightly worse than theirs.

from async-rl.

@muupan
Thank you for sharing implementation and setting with great result!

Your wiki helps a lot, and I'm going to try your setting.

Let me ask you something not written in wiki.

1. There is loss normalization code for when sequence terminated at the middle

https://github.com/muupan/async-rl/blob/master/a3c.py#L113-L118

Are you using this now?

3. There is an action skipping code at ALE # initialize()

https://github.com/muupan/async-rl/blob/master/ale.py#L146-L149

What is this for?

And I'm going to adjust my parameter as written in your wiki. Thanks!!

from async-rl.

1. No, I don't use it now.
2. It is called "no-op max" in the Nature DQN paper. It adds some randomness to initial states.

from async-rl.

I see. Thank you!

from async-rl.