uber-research / deep-neuroevolution Goto Github PK

View Code? Open in Web Editor NEW

1.6K 80.0 298.0 3.71 MB

Deep Neuroevolution

License: Other

Python 82.64% Shell 1.07% Makefile 0.32% C++ 15.83% Dockerfile 0.13%

deep-neuroevolution ai machine-learning reinforcement-learning

deep-neuroevolution's People

Contributors

Stargazers

Watchers

Forkers

kastnerkyle codeaudit project-cidy dl-yc cantren amoliu ggdupont 10183308 kashif mrgoogol liyuanyaun little1tow ml-lab freephys guilherme-pombo goldenzero spsancti deeplearningupc karyafar fitrialif jdc08161063 nethask onisimchukv johmedr jiths keithsw leishuyu-bupt jcrompton mrinalarun prosperityai shubhampachori12110095 vashishtmadhavan jackyzhougithub qq1074123922 sunnyangzs cgebe xu3kev neko0 kelvinson hedgefair qzhong0605 pondercode zenkijn frankatmech akbek datapresso b-xiang nwpark parrondo bssrdf ladin157 qitong leandroagudelo189 riseooi norcimo5 wrmsr mazecreator cpehle longjohncoder markovchainmontecarlo phimachine anands09 laal65 abpin iywo dzugaru ituco hechong-dev dewival nostrademous harunahaju ogata2606 jiangziguo neerajprad bin913 highwayns kenqyu daghty feiga hushuitian jreuben11 abdelmojeb mechanicalai pb-pravin tickpeach mohisen dreadlord1984 jianshijim lalalland tesserai pandinosaurus fcccode xiarixiaoyao minchaokang fundou leesy714 liorfuks augmen bmw111 hzyjerry

deep-neuroevolution's Issues

Undefined names 'mujoco' and 'SubprocVecEnv'

flake8 testing of https://github.com/uber-common/deep-neuroevolution on Python 3.6.3

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./gpu_implementation/gym_tensorflow/__init__.py:13:16: F821 undefined name 'mujoco'
        return mujoco.HumanoidMO('Humanoid-v1', batch_size, *args, **kwargs)
               ^
./gpu_implementation/gym_tensorflow/__init__.py:15:16: F821 undefined name 'SubprocVecEnv'
        return SubprocVecEnv([lambda: make(game[4:], 1, *args, **kwargs) for _ in range(batch_size)])
               ^
2     F821 undefined name 'mujoco'
2

proposed fix for nan error in vine

in visual_inspector/figure_base/load_data.py

the line:

cind = (fitness - minfit)/(maxfit - minfit) * gs.numBins

will result in a nan error when maxfit == minfit

I suggest:

if maxfit == minfit:
    cind = 0.0
else:    
    cind = (fitness - minfit)/(maxfit - minfit) * gs.numBins

Local experiment run issue

I lunch any of the local experiment after setting up redis server for master and relay.
I get stuck in iteration 0 for the master while the relay keep retrying
2018-11-19 15:07:39,067 pid=31344] [master] Declared experiment {'config': {'calc_obstat_prob': 0.0,
'episode_cutoff_mode': 5000, 'episodes_per_batch': 5000, 'eval_prob': 0.01, 'l2coeff': 0.005, 'noise_stdev': 0.005, 'return_proc_mode': 'centered_rank', 'snapshot_freq': 20, 'timesteps_per_batch': 10000}, 'env_id': 'FrostbiteNoFrameskip-v4', 'num_elites': 1, 'policy': {'args': {'nonlin_type': 'relu'}, 'type': 'GAAtariPolicy'}, 'population_size': 10} ********** Iteration 0 **********
[2018-11-19 15:10:58,121 pid=31372] es:exp not set. Retrying after 4.89 sec (41/300)
I am not sure why the workers cannot connect to the master.

Final output after running the code

Minor issues in GA experiment

Issue 1:
At line:

deep-neuroevolution/es_distributed/ga.py

Line 230 in 6ab22e1

 eval_rews, eval_length = policy.rollout(env) # eval rollouts don't obey task_data.timestep_limit 

The rollout function returns 3 objects, due to which we getting an error:
ValueError: too many values to unpack (expected 2)
It could be solved by replacing line 230 with:
eval_rews, eval_length, _ = policy.rollout(env)
Issue 2:
At line:

deep-neuroevolution/es_distributed/ga.py

Line 143 in 6ab22e1

lengths_n2 = np.array([r.lengths_n2 for r in curr_task_results])

It should be lengths_n2 = np.concatenate([r.lengths_n2 for r in curr_task_results]) very much similar to

deep-neuroevolution/es_distributed/es.py

Line 276 in 6ab22e1

lengths_n2 = np.concatenate([r.lengths_n2 for r in curr_task_results])

Otherwise lengths_n2.mean() and lengths_n2.sum() will have multiple dimension.

Run Gym.CartPole-v1

Hi,
I am trying to run gpu-implementation es.py using gym enviroment gym.CartPole-v1
I edited the configuration file es_atari_config.json by changing "frostbrite" to "gym.CartPole-V1" and encounter error below

2018-08-18 12:09:54.669655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
08/18/2018 12:09:54 PM {
    "episode_cutoff_mode": 5000,
    "game": "gym.CartPole-v1",
    "l2coeff": 0.005,
    "model": "ModelVirtualBN",
    "mutation_power": 0.02,
    "num_test_episodes": 200,
    "num_validation_episodes": 30,
    "optimizer": {
        "args": {
            "stepsize": 0.01
        },
        "type": "adam"
    },
    "population_size": 5000,
    "return_proc_mode": "centered_rank",
    "timesteps": 250000000.0
}
08/18/2018 12:09:54 PM Logging to: /tmp/tmp59hdhjso
2018-08-18 12:09:54.745411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 12:09:54.745453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 12:09:54.745472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-08-18 12:09:54.745476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-08-18 12:09:54.745588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
Traceback (most recent call last):
  File "es.py", line 293, in <module>
    main(**exp)
  File "es.py", line 148, in main
    worker = ConcurrentWorkers(make_env, Model, batch_size=64)
  File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/neuroevolution/concurrent_worker.py", line 135, in __init__
    ref_batch = gym_tensorflow.get_ref_batch(make_env_f, sess, 128)
  File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/gym_tensorflow/__init__.py", line 19, in get_ref_batch
    assert env.discrete_action
AssertionError

I notice that the GymEnv in tf_env.py having property of discrete_action that will only return False which is triggering the assert in the get_ref_batch

class GymEnv(PythonEnv): ... @property def discrete_action(self): return False

Is openai gym's enviroment ready for run?

Speed comparisons and Muzero?

Are there any speed benchmarks with the GPU implementation and other popular methods of training AI's such as Muzero?

Also would there be any advantage in paring this with Muzero?

ranking is not done correctly

In ES, compute_ranks() does an argsort, which will give different ranks to individuals with the same fitness.
This introduces a noise in the gradient estimate. This is not a big issue since the expected value of the noise is zero, but it can slow down convergence.
This is really only a problem in environments, where rewards are sparse, so a lot of individuals will have the same fitness.
Solution: Average ranks for individuals with equal fitness.

Data format for dimensionality reduction [Atari domain]

Hello,
To be able to visualize the cloud plot using vine for the Atari domain, you said that we need to capture the final emulator RAM state ( integer-valued vectors of length 128 that capture all the state variables in a game) as the BC, I am wondering what format of data should we be entering as input to the "process_bc" script for dimensionality reduction ?
Thank you!

Minor Typo

You have a minor typo in:
gpu_implementation/README.md

This:
python ga.py ra_atari_config.json
Should be this:
python ga.py rs_atari_config.json

Unless the filename is mis-named.

Unable to launch sample GA and RS experiment

I have got the following error message when I tried to launch sample GA experiment:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/andi/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/andi/deep-neuroevolution/es_distributed/main.py", line 84, in workers
    algo.run_worker(master_redis_cfg, relay_redis_cfg, noise=noise)
  File "/home/andi/deep-neuroevolution/es_distributed/ga.py", line 267, in run_worker
    policy, env, task_data.timestep_limit, rs, task_ob_stat, config.calc_obstat_prob)
  File "/home/andi/deep-neuroevolution/es_distributed/ga.py", line 29, in rollout_and_update_ob_stat
    rollout_rews, rollout_len = policy.rollout(env, timestep_limit=timestep_limit, random_stream=rs)
ValueError: too many values to unpack (expected 2)

and the following error message when trying to launch sample RS experiment:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/andi/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/andi/deep-neuroevolution/env/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/andi/deep-neuroevolution/es_distributed/main.py", line 61, in master
    algo.run_master({'unix_socket_path': master_socket_path}, log_dir, exp)
  File "/home/andi/deep-neuroevolution/es_distributed/rs.py", line 52, in run_master
    timestep_limit=tslimit
TypeError: __new__() missing 1 required positional argument: 'ref_batch'

Thanks

tmux command not found

Hi, I ran into trouble when running the command ". scripts/local_run_exp.sh", where the cygwin terminal gives "-bash: tmux: command not found". Seems like nobody have ran into this problem before, I hope someone can give a help. Thanks in advance!

Building gym_tensorflow

Getting the following errors from a fresh git clone following README instructions:
SIDE NOTE: I'm installing this on an Ubuntu OS using Windows Subsystem for Linux

(env) nostrademous@DESKTOP-J9431IB:~/ML/deep-neuroevolution/gpu_implementation/gym_tensorflow$ make clean
rm -rf gym_tensorflow.so
(env) nostrademous@DESKTOP-J9431IB:~/ML/deep-neuroevolution/gpu_implementation/gym_tensorflow$ make
g++ -std=c++11 -shared -fPIC -I/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include -I/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/external/nsync/public -L/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/core -D_GLIBCXX_USE_CXX11_ABI=0 -O2 -DGOOGLE_CUDA=1 -Wl,-rpath=/build .//*.cpp .//ops/*.cpp -ltensorflow_framework -o gym_tensorflow.so
In file included from .//tf_env.cpp:22:0:
.//tf_env.cpp: In member function ‘virtual void EnvironmentMakeOp::Compute(tensorflow::OpKernelContext*)’:
.//tf_env.cpp:102:69: error: ‘MakeResourceHandleToOutput’ was not declared in this scope
                                     MakeTypeIndex<BaseEnvironment>()));
                                                                     ^
/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:1309:29: note: in definition of macro ‘OP_REQUIRES_OK’
     ::tensorflow::Status _s(STATUS);    \
                             ^
In file included from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/stream_executor/platform/mutex.h:25:0,
                 from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/stream_executor/dso_loader.h:29,
                 from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/platform/default/stream_executor.h:25,
                 from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/platform/stream_executor.h:24,
                 from .//ops/indexedmatmul.cpp:20:
/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/stream_executor/platform/default/mutex.h:32:19: error: ‘tensorflow::tf_shared_lock’ has not been declared
 using tensorflow::tf_shared_lock;
                   ^
In file included from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/platform/default/stream_executor.h:31:0,
                 from /home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/platform/stream_executor.h:24,
                 from .//ops/indexedmatmul.cpp:20:
/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/stream_executor/stream.h: In member function ‘bool perftools::gputools::Stream::InErrorState() const’:
/home/nostrademous/ML/env/lib/python3.6/site-packages/tensorflow/include/tensorflow/stream_executor/stream.h:2005:5: error: ‘tf_shared_lock’ was not declared in this scope
     tf_shared_lock lock{mu_};
     ^
Makefile:45: recipe for target 'gym_tensorflow.so' failed
make: *** [gym_tensorflow.so] Error 1

NOTE: I do have slightly update version of some of the python packages, but I don't think that's the errors I'm hitting. Here is the pip list anyways:

(env) nostrademous@DESKTOP-J9431IB:~/ML/deep-neuroevolution/gpu_implementation/gym_tensorflow$ pip list
Package        Version
-------------- -----------
absl-py        0.2.0
appdirs        1.4.3
astor          0.6.2
bleach         1.5.0
click          6.7
gast           0.2.0
grpcio         1.11.0
gym            0.9.4
h5py           2.7.0
html5lib       0.9999999
Markdown       2.6.11
mujoco-py      0.5.7
numpy          1.14.3
packaging      16.8
pip            10.0.1
protobuf       3.5.2.post1
pyglet         1.2.4
PyOpenGL       3.1.0
pyparsing      2.2.0
redis          2.10.5
requests       2.14.2
setuptools     28.8.0
six            1.11.0
tensorboard    1.8.0
tensorflow     0.12.1
tensorflow-gpu 1.8.0
termcolor      1.1.0
Werkzeug       0.14.1
wheel          0.31.0
(env) nostrademous@DESKTOP-J9431IB:~/ML/deep-neuroevolution/gpu_implementation/gym_tensorflow$

module 'gym' has no attribute 'undo_logger_setup'

module 'gym' has no attribute 'undo_looger_setup'

When I run '. scripts/local_run_exp.sh ga configurations/frostbite_ga.json', it occur that error.

AttributeError: 'list' object has no attribute 'ndim'

When I run '. scripts/local_run_exp.sh es configurations/frostbite_es.json', I got such error 'AttributeError: 'list' object has no attribute 'ndim'' .

Part of log:
'env_id': 'FrostbiteNoFrameskip-v4',
'optimizer': {'args': {'stepsize': 0.01}, 'type': 'adam'},
'policy': {'args': {}, 'type': 'ESAtariPolicy'}}
********** Iteration 0 **********
Traceback (most recent call last):
File "/home/gaoli/tools/Python3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/gaoli/tools/Python3.6/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/gaoli/CODE/deep-neuroevolution/es_distributed/main.py", line 90, in
cli()
File "/home/gaoli/CODE/deep-neuroevolution/env/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/home/gaoli/CODE/deep-neuroevolution/env/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/gaoli/CODE/deep-neuroevolution/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gaoli/CODE/deep-neuroevolution/env/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/gaoli/CODE/deep-neuroevolution/env/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/gaoli/CODE/deep-neuroevolution/es_distributed/main.py", line 61, in master
algo.run_master({'unix_socket_path': master_socket_path}, log_dir, exp)
File "/home/gaoli/CODE/deep-neuroevolution/es_distributed/es.py", line 246, in run_master
assert (result.noise_inds_n.ndim == 1 and
AttributeError: 'list' object has no attribute 'ndim'

"timesteps" meaning in config file?

Hi, what is the meaning of "timesteps" parameters in ga_atari_config.json? Is it the total frames or Forward Passes in the paper? Thanks.

Novelty Search for gpu implementation

I notice Novelty Seeking is not implemented for gpu implementation.
Is there any issue that might block Novelty Seeking implementation using gpu?

Dockerfile for gpu implementation?

Anyone have a working dockerfile for the gpu implementation?

"No ROM File specified or the ROM file was not found"

06/09/2019 08:36:22 PM {
"episode_cutoff_mode": 5000,
"game": "frostbite",
"l2coeff": 0.005,
"model": "ModelVirtualBN",
"mutation_power": 0.02,
"num_test_episodes": 200,
"num_validation_episodes": 30,
"optimizer": {
"args": {
"stepsize": 0.01
},
"type": "adam"
},
"population_size": 5000,
"return_proc_mode": "centered_rank",
"timesteps": 250000000.0
}
06/09/2019 08:36:22 PM Logging to: /tmp/tmphfvloc5w
2019-06-09 20:36:22.309953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2019-06-09 20:36:22.310085: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 73 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
No ROM File specified or the ROM file was not foun

I got this problem when runing es.py with the config file es_atari_config.json. anyone else encountered the same issue? It seems that the progaram has successfully loaded the config file, but failed to load rom file.

error while running ga experiment

so while trying to run the ga code for frostbite environment I run into this issue
ValueError: kth(=-5) out of bounds (5)
the source of error is this
idx = np.argpartition(returns_n2, (-population_size, -1))[-1:-population_size-1:-1]

Anybody knows how to resolve this?

Worker receives old task_id from pipe

WorkerClient.get_current_task() doesn't return the new task_id when the master's work_id changes, but instead keeps returning the old task_id. The workers cached_task_id is thus never updated.
What can be the reason for this?

[es_distributed/tf_util.py] ValueError: Dimension 0 in both shapes must be equal, but are 4608 and 18. Shapes are [4608] and [18]. From merging shape 12 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [4096], [16], [16], [16], [8192], [32], [32], [32], [991232], [ 256], [256], [256], [4608], [18].

Hello People,

I managed to find some error, when I tested the ES algorithm.

python3 -m es_distributed.main master --master_socket_path /tmp/es_redis_master.sock --algo es --exp_file configurations/frostbite_es.json                       [67/67]
file configurations/frostbite_es.json /tmp/es_redis_master.sock --algo es --exp_f                                                                                       
[2019-06-04 23:15:37,056 pid=22170] run_master: {'exp': {'config': {'calc_obstat_prob': 0.0, 'episodes_per_batch': 5000, 'eval_prob': 0.01, 'l2coeff': 0.005, 'noise_std
ev': 0.005, 'snapshot_freq': 20, 'timesteps_per_batch': 10000, 'return_proc_mode': 'centered_rank', 'episode_cutoff_mode': 5000}, 'env_id': 'FrostbiteNoFrameskip-v4', '
optimizer': {'args': {'stepsize': 0.01}, 'type': 'adam'}, 'policy': {'args': {}, 'type': 'ESAtariPolicy'}}, 'log_dir': '/tmp/es_master_22170', 'master_redis_cfg': {'uni
x_socket_path': '/tmp/es_redis_master.sock'}}                                                                                                                           
[2019-06-04 23:15:38,083 pid=22170] Tabular logging to /tmp/es_master_22170                                                                                             
2019-06-04 23:15:38.894940: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3410300000 Hz                                                      
2019-06-04 23:15:38.895374: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x40bfa50 executing computations on platform Host. Devices:                   
2019-06-04 23:15:38.895393: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>                                     
[2019-06-04 23:15:38,904 pid=22170] From /home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorfl
ow.python.framework.ops) is deprecated and will be removed in a future version.                                                                                         
Instructions for updating:                                                                                                                                              
Colocations handled automatically by placer.                                                                                                                            
[2019-06-04 23:15:38,991 pid=22170] From /home/dragon/.local/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py:1624: flatten (from tensorflo
w.python.layers.core) is deprecated and will be removed in a future version.                                                                                            
Instructions for updating:                                                                                                                                              
Use keras.layers.flatten instead.                                                                                                                                       
[2019-06-04 23:15:39,054 pid=22170] From /home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/util/decorator_utils.py:145: GraphKeys.VARIABLES (from tensor
flow.python.framework.ops) is deprecated and will be removed in a future version.                                                                                       
Instructions for updating:                                                                                                                                              
Use `tf.GraphKeys.GLOBAL_VARIABLES` instead.
Traceback (most recent call last):                                                                                                                               [45/67]
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 4608 and 18. Shapes are [4608] and [18].
        From merging shape 12 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [4096], [16], [16], [16], [8192], [32], [32], [32], [991232], [
256], [256], [256], [4608], [18].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 956, in invoke                                                                      [23/67]
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 61, in master
    algo.run_master({'unix_socket_path': master_socket_path}, log_dir, exp)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/es.py", line 147, in run_master
    config, env, sess, policy = setup(exp, single_threaded=False)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/es.py", line 136, in setup
    policy = getattr(policies, exp['policy']['type'])(env.observation_space, env.action_space, **exp['policy']['args'])
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/policies.py", line 24, in __init__
    self._getflat = U.GetFlat(self.trainable_variables)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/tf_util.py", line 244, in __init__ 
    self.op = tf.concat(0, [tf.reshape(v, [numel(v)]) for v in var_list])                                     
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1253, in concat
    dtype=dtypes.int32).get_shape().assert_is_compatible_with(
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1039, in convert_to_tensor
    return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1097, in convert_to_tensor_v2
    as_ref=False)
File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1175, in internal_convert_to_tensor                                   
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1102, in _autopacking_conversion_function
    return _autopacking_helper(v, dtype, name or "packed")
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1054, in _autopacking_helper
    return gen_array_ops.pack(elems_as_tensors, name=scope)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 5448, in pack
    "Pack", values=values, axis=axis, name=name)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimension 0 in both shapes must be equal, but are 4608 and 18. Shapes are [4608] and [18].
From merging shape 12 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [4096], [16], [16], [16], [8192], [32], [32], [32], [991232], [
256], [256], [256], [4608], [18].

Most of these errors were related to es_distributed/tf_util.py file which was originated from the tf.concat function or method.

Below were some of the changes:

def concatenate(arrs, axis=0) function at line 30 - 31

from:

def concatenate(arrs, axis=0):
    return tf.concat(axis, arrs)

to:

def concatenate(arrs, axis=0):
    return tf.concat(arrs, axis)

def flatgrad(loss, var_list) function at line 219 - 222

from:

def flatgrad(loss, var_list):
    grads = tf.gradients(loss, var_list)
    return tf.concat(0, [tf.reshape(grad, [numel(v)])
        for (v, grad) in zip(var_list, grads)])

to:

def flatgrad(loss, var_list):
    grads = tf.gradients(loss, var_list)
    return tf.concat([tf.reshape(grad, [numel(v)], 0)
        for (v, grad) in zip(var_list, grads)])

def __init__(self, var_list) function at line 243 -244

from:

def __init__(self, var_list):
       self.op = tf.concat(0, [tf.reshape(v, [numel(v)]) for v in var_list])

to:

def __init__(self, var_list):
       self.op = tf.concat([tf.reshape(v, [numel(v)]) for v in var_list], 0)

My environment information:
Ubuntu 18.04 x64
Python 3.6.8
tensorflow 1.13.1
Click 7.0
atari-py 0.1.15
numpy 1.16.3
gym 0.12.1
baselines 0.1.5

Thanks

how to run locally

'es:exp not set'

I have been getting this error, upon running the command,
. scripts/local_run_exp.sh es configurations/frostbite_es.json.

Snapshots visualization

When I try to visualize the snapshots generated by games like Frostbite, i just run the command

python -m scripts.viz 'FrsotbiteNoFrameskip-v0' <snapshot_file>

This works like a charm, but when I change the game, so like SpaceInvaders, after the training and the generation of the snapshots files, when I run:

python -m scripts.viz 'SpaceInvadersNoFrameskip-v0' <snapshot_file>

It doesn't work as it expects inputs of shape (1,84,84,4) but SpaceInvaders' inputs are (1, 210, 160, 3).

Its like the script viz.py has stubbed the expected input shape and can't handle different games from those with (1,84,84,4) tensors.

Did anyone run into the same issue?

Details of the error:
Input command:

python3 -m scripts.viz 'SpaceInvadersNoFrameskip-v0' keep_exploring/old_gens/gen_125-153/snapshot_iter00028_rew1100.h5

Error reported:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/scripts/viz.py", line 63, in <module>
    main()
  File "/home/emanuele/.local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/emanuele/.local/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/emanuele/.local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/emanuele/.local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/scripts/viz.py", line 54, in main
    rews, t, novelty_vector = pi.rollout(env, render=True, random_stream=np.random if stochastic else None)
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/es_distributed/policies.py", line 509, in rollout
    ac = self.act(ob[None], random_stream=random_stream)[0]
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/es_distributed/policies.py", line 485, in act
    return self._act(train_vars)
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/es_distributed/tf_util.py", line 176, in <lambda>
    return lambda *inputs : f(*inputs)[0]
  File "/home/emanuele/Desktop/deepneuro/deep-neuroevolution/es_distributed/tf_util.py", line 191, in __call__
    results = get_session().run(self.outputs_update, feed_dict=feed_dict)[:-1]
  File "/home/emanuele/.local/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 950, in run
    run_metadata_ptr)
  File "/home/emanuele/.local/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1149, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 84, 84, 4) for Tensor 'GAAtariPolicy/Placeholder:0', which has shape '(?, 210, 160, 3)'

Weight Transfer into Tensorflow Model

Hello,

First of all, I think you did some really great work! I am trying to apply the idea of using GAs to train models for some of my own work and have been looking to your work and code for some inspiration. I am wondering if you could help me find out how the weight calculated by the GA are fed into the tensorflow model archictecture? I have been looking through the repo for a while but am struggling to find the link between the GA and the model run.

Thanks!

no file or directory '/tmp/es_redis_master.sock.'

I got the following error:

[2019-07-03 17:08:04,453 pid=11687] Could not connect to {'unix_socket_path': '/tmp/es_redis_master.sock'}. Retrying after 7.11 sec (2/300). Error: Error 2 connecting to unix socket: /tmp/es_redis_master.sock. No such file or directory.

Could you please tell me how to resolve it? Thanks.

[Local ES Frostbite] Value Error, Dimension Mismatch

When running
. scripts/local_run_exp.sh es configurations/frostbite_es.json

I get a Value Error/Dimension Mismatch. Here is the full output of the redis master:

. scripts/local_env_setup.sh                                                                                                                                                                                 [29/29]
python -m es_distributed.main master --master_socket_path /tmp/es_redis_master.sock --algo es --exp_file configurations/frostbite_es.json
marvin@mlpad:~/code/deep-neuroevolution$ . scripts/local_env_setup.sh
Setting up local environment
(env) marvin@mlpad:~/code/deep-neuroevolution$ python -m es_distributed.main master --master_socket_path /tmp/es_redis_master.sock --algo es --exp_file configurations/frostbite_es.json
[2019-06-04 18:12:12,474 pid=5654] run_master: {'master_redis_cfg': {'unix_socket_path': '/tmp/es_redis_master.sock'}, 'log_dir': '/tmp/es_master_5654', 'exp': {'config': {'calc_obstat_prob': 0.0, 'episodes_per_$
atch': 5000, 'eval_prob': 0.01, 'l2coeff': 0.005, 'noise_stdev': 0.005, 'snapshot_freq': 20, 'timesteps_per_batch': 10000, 'return_proc_mode': 'centered_rank', 'episode_cutoff_mode': 5000}, 'env_id': 'FrostbiteN$
Frameskip-v4', 'optimizer': {'args': {'stepsize': 0.01}, 'type': 'adam'}, 'policy': {'args': {}, 'type': 'ESAtariPolicy'}}}
[2019-06-04 18:12:13,773 pid=5654] Tabular logging to /tmp/es_master_5654
2019-06-04 18:12:14.669307: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-06-04 18:12:14.700165: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2496000000 Hz
2019-06-04 18:12:14.700767: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x3ad73d0 executing computations on platform Host. Devices:
2019-06-04 18:12:14.700785: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
[2019-06-04 18:12:14,711 pid=5654] From /home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops
) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
[2019-06-04 18:12:14,854 pid=5654] From /home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/contrib/layers/python/layers/layers.py:1624: flatten (from tensorflow.python.layers.core) i
s deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
[2019-06-04 18:12:14,939 pid=5654] From /home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/util/decorator_utils.py:145: GraphKeys.VARIABLES (from tensorflow.python.framework.o
ps) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.GraphKeys.GLOBAL_VARIABLES` instead.
Traceback (most recent call last):
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 4608 and 18. Shapes are [4608] and [18].
	From merging shape 12 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [4096], [16], [16], [16], [8192], [32], [32], [32], [991232], [256], [256], [256], [4608], [18].

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/marvin/code/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/marvin/code/deep-neuroevolution/es_distributed/main.py", line 61, in master
    algo.run_master({'unix_socket_path': master_socket_path}, log_dir, exp)
  File "/home/marvin/code/deep-neuroevolution/es_distributed/es.py", line 147, in run_master
    config, env, sess, policy = setup(exp, single_threaded=False)
  File "/home/marvin/code/deep-neuroevolution/es_distributed/es.py", line 136, in setup
    policy = getattr(policies, exp['policy']['type'])(env.observation_space, env.action_space, **exp['policy']['args'])
  File "/home/marvin/code/deep-neuroevolution/es_distributed/policies.py", line 24, in __init__
    self._getflat = U.GetFlat(self.trainable_variables)
  File "/home/marvin/code/deep-neuroevolution/es_distributed/tf_util.py", line 244, in __init__
    self.op = tf.concat(0, [tf.reshape(v, [numel(v)]) for v in var_list])
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 1253, in concat
    dtype=dtypes.int32).get_shape().assert_is_compatible_with(
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1039, in convert_to_tensor
    return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1097, in convert_to_tensor_v2
    as_ref=False)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1175, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 1102, in _autopacking_conversion_function
    return _autopacking_helper(v, dtype, name or "packed")
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 1054, in _autopacking_helper
    return gen_array_ops.pack(elems_as_tensors, name=scope)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 5448, in pack
    "Pack", values=values, axis=axis, name=name)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/home/marvin/code/deep-neuroevolution/env/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimension 0 in both shapes must be equal, but are 4608 and 18. Shapes are [4608] and [18].
	From merging shape 12 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [4096], [16], [16], [16], [8192], [32], [32], [32], [991232], [256], [256], [256], [4608], [18].

How do I fix this and get the example up and running? I'd be grateful for any help!

Cheers,
Marvin

Elitism

Hi,

It is not entirely clear to me how you guys handle elitism. In your paper (link) you mention "The Nth individual is an unmodified copy of the best individual from the previous generation,...". However, if I understand correctly, in the ga and ga_modified algorithms your workers still evolve the parameters if they sample the first (or last) individual in the population.

In fact I don't immediately see where any of the mentioned 'evaluate top 10 individuals from previous generation on 30 additional episodes' takes place.

I'm just curious because I'm not sure how to handle elitism in a project myself. Thanks in advance :)

[es_distributed/rs.py] eval_rews, eval_length = policy.rollout(env) ValueError: too many values to unpack (expected 2)

Hello People,

I am getting the following error when I run . scripts/local_run_exp.sh rs configurations/frostbite_ga.json, which is the RS experiment

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 84, in workers
    algo.run_worker(master_redis_cfg, relay_redis_cfg, noise=noise)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/rs.py", line 190, in run_worker
    eval_rews, eval_length = policy.rollout(env)  # eval rollouts don't obey task_data.timestep_limit
ValueError: too many values to unpack (expected 2)

and I managed to fixed it by changing the following code:

from

deep-neuroevolution/es_distributed/rs.py, line 190:

eval_rews, eval_length = policy.rollout(env) # eval rollouts don't obey task_data.timestep_limit

to:

deep-neuroevolution/es_distributed/rs.py, line 190:

eval_rews, eval_length, _ = policy.rollout(env) # eval rollouts don't obey task_data.timestep_limit

My environment information:
Ubuntu 18.04 x64
Python 3.6.8
tensorflow 1.13.1
Click 7.0
atari-py 0.1.15
numpy 1.16.3
gym 0.12.1
baselines 0.1.5

Thanks

how to run gpu_implementation on GPU

When I used nvidia 1080ti, I was able to compile gym_tensorflow.so and run the exp. The env is tensorflow-gpu 1.8.0 and cuda version is 9.0. But when I switch to 2080ti, the exp run into trouble as follow:

2019-06-02 18:03:10.727082: E tensorflow/stream_executor/cuda/cuda_blas.cc:654] failed to run cuBLAS routine cublasSgemmBatched: CUBLAS_STATUS_EXECUTION_FAILED
2019-06-02 18:03:10.727109: E tensorflow/stream_executor/cuda/cuda_blas.cc:2413] Internal: failed BLAS call, see log for details
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1322, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Blas xGEMMBatched launch failed : a.shape=[1,441,256], b.shape=[64,256,16], idx.shape=[1], m=441, n=16, k=256, batch_size=1
[[Node: model/Model/conv1/IndexedBatchMatMul = IndexedBatchMatMul[T=DT_FLOAT, adj_x=false, adj_y=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](model/Model/conv1/Reshape_2, model/Model/conv1/Reshape, _arg_model/Placeholder_0_0/_37)]]
[[Node: model/Identity_1/_59 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_210_model/Identity_1", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/root/hz/deep-neuroevolution/gpu_implementation/neuroevolution/concurrent_worker.py", line 94, in _loop
rews, is_done, _ = self.sess.run([self.rew_op, self.done_op, self.incr_counter], {self.placeholder_indices: indices})
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)

I tried to upgrade tensorflow-gpu version to 1. 12,1.13, but under that env , gym_tensorflow.so could not be compiled. I also found a similar issue in https://github.com/qqwweee/keras-yolo3/issues/332 , but still no luck after I take action to Install patchs for cuda9,
I wonder if there is a solution.

[es_distributed/ga.py] eval_rews, eval_length = policy.rollout(env) ValueError: too many values to unpack (expected 2)

Hello People,

I am getting the following error when I run . scripts/local_run_exp.sh ga configurations/frostbite_ga.json, which is the GA experiment

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 90, in <module>
    cli()
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dragon/.local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/main.py", line 84, in workers
    algo.run_worker(master_redis_cfg, relay_redis_cfg, noise=noise)
  File "/home/dragon/machine_learning/deep-neuroevolution/es_distributed/ga.py", line 230, in run_worker
    eval_rews, eval_length = policy.rollout(env)  # eval rollouts don't obey task_data.timestep_limit
ValueError: too many values to unpack (expected 2)

and I managed to fixed it by changing the following code:

from

deep-neuroevolution/es_distributed/ga.py, line 230:

eval_rews, eval_length = policy.rollout(env) # eval rollouts don't obey task_data.timestep_limit

to:

deep-neuroevolution/es_distributed/ga.py, line 230:

eval_rews, eval_length, _ = policy.rollout(env) # eval rollouts don't obey task_data.timestep_limit

My environment information:
Ubuntu 18.04 x64
Python 3.6.8
tensorflow 1.13.1
Click 7.0
atari-py 0.1.15
numpy 1.16.3
gym 0.12.1
baselines 0.1.5

Thanks

Do not move

Thank you for opening code.
Along with your Readme.md, I did

. scripts/local_run_redis.sh

error happens

*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 80

'protected-mode yes'
Bad directive or wrong number of arguments

What is protected-mode? and what should I do to do program?

uber-research / deep-neuroevolution Goto Github PK

deep-neuroevolution's People

Contributors

Stargazers

Watchers

Forkers

deep-neuroevolution's Issues

Recommend Projects

Recommend Topics

Recommend Org