Hi, I've downloaded the code and try to run it. But I cannot get the same result as yours, it actually never converges. To make the code working, I only commented out below code in REINFORCE.jl
, because Flux.Tracker
is not supported anymore and CircularBuffer
seems not used. Other than that nothing is changed.
...
Episode 4987 = 21.0 | Mean of the last 10 episodes = 22.9
Episode 4988 = 24.0 | Mean of the last 10 episodes = 23.5
Episode 4989 = 40.0 | Mean of the last 10 episodes = 26.3
Episode 4990 = 18.0 | Mean of the last 10 episodes = 27.2
Episode 4991 = 26.0 | Mean of the last 10 episodes = 27.5
Episode 4992 = 37.0 | Mean of the last 10 episodes = 28.6
Episode 4993 = 20.0 | Mean of the last 10 episodes = 27.4
Episode 4994 = 25.0 | Mean of the last 10 episodes = 28.6
Episode 4995 = 27.0 | Mean of the last 10 episodes = 26.5
Episode 4996 = 33.0 | Mean of the last 10 episodes = 27.1
Episode 4997 = 12.0 | Mean of the last 10 episodes = 26.2
Episode 4998 = 31.0 | Mean of the last 10 episodes = 26.9
Episode 4999 = 13.0 | Mean of the last 10 episodes = 24.2
Episode 5000 = 15.0 | Mean of the last 10 episodes = 23.9
julia> versioninfo()
Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin18.7.0)
CPU: Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 2
(@v1.4) pkg> st
Status `~/.julia/environments/v1.4/Project.toml`
[c52e3926] Atom v0.12.11
[336ed68f] CSV v0.6.2
[a93c6f00] DataFrames v0.21.0
[31c24e10] Distributions v0.23.2
[587475ba] Flux v0.10.4
[56b9baea] Gym v0.1.0 #master (https://github.com/FluxML/Gym.jl)
[e5e0dc1b] Juno v0.8.2
[7f8f8fb0] LearnBase v0.4.1
[91a5bcdd] Plots v1.3.1
[438e738f] PyCall v1.91.4
[0376cc21] Reinforce v0.3.0
[2913bbd2] StatsBase v0.33.0
[4c63d2b9] StatsFuns v0.9.4
[9a3f8284] Random
[10745b16] Statistics