Can't render atari games env on google collab

I am not able to render the Atari games env on google collaboratory. Any specific reasons why it doesn't support it? I am able to run it on jupyter notebook or any other IDE like pycharm but can't on collab. Just curious to know more about it.

Details:

OS : Windows11
OpenAI Gym : version 0.21
Package : Gym[atari]
Tensorflow version : 2.8.0
keras-rl2 version : 1.0.5
CUDA Version : 11.2
GPU Name : Tesla T4
NVIDIA-SMI 460.32.03

'module' object has no attribute 'pb'

File "/home/sdb/618/anaconda3/envs/py2.7/lib/python2.7/site-packages/tensorboard/summary/v1.py", line 46, in
pr_curve_pb = _pr_curve_summary.pb

Low Score

Hey Greg!

I tried running your program for Breakout. My output score for training and testing came out to be really low.

Could you guide me on how should I improve the agent, or where could I be going wrong?

Thanks!

e Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW

I am running this project into my ubuntu VM with tensorflow CPU version available. Getting following error. Can you please help me on this? HOw i can run this into my laptop?

{"metric": "run", "value": 290}
Traceback (most recent call last):
File "/home/akayal/amit/atari-master/atari.py", line 110, in
Atari()
File "/home/akayal/amit/atari-master/atari.py", line 30, in init
self._main_loop(self._game_model(game_mode, game_name, env.action_space.n), env, render, total_step_limit, total_run_limit, clip)
File "/home/akayal/amit/atari-master/atari.py", line 65, in _main_loop
game_model.step_update(total_step)
File "/home/akayal/amit/atari-master/game_models/ddqn_game_model.py", line 101, in step_update
loss, accuracy, average_max_q = self._train()
File "/home/akayal/amit/atari-master/game_models/ddqn_game_model.py", line 130, in _train
next_state_prediction = self.ddqn_target.predict(next_state)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 87, in _method_wrapper
return method(self, *args, **kwargs)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1203, in predict
tmp_batch_outputs = predict_function(iterator)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 580, in call
result = self._call(*args, **kwds)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 650, in _call
return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call
return self._call_flat(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 593, in call
outputs = execute.execute(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: The Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW
[[node sequential_1/conv2d_3/Conv2D (defined at /amit/atari-master/game_models/ddqn_game_model.py:130) ]] [Op:__inference_predict_function_427]

Function call stack:
predict_function
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call
return self._call_flat(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 593, in call
outputs = execute.execute(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: The Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW
[[node sequential_1/conv2d_3/Conv2D (defined at /amit/atari-master/game_models/ddqn_game_model.py:130) ]] [Op:__inference_predict_function_427]

Function call stack:
predict_function
File "/home/akayal/amit/atari-master/game_models/ddqn_game_model.py", line 130, in _train
next_state_prediction = self.ddqn_target.predict(next_state)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 87, in _method_wrapper
return method(self, *args, **kwargs)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1203, in predict
tmp_batch_outputs = predict_function(iterator)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 580, in call
result = self._call(*args, **kwds)
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 650, in _call
return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call
return self._call_flat(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 593, in call
outputs = execute.execute(
File "/home/akayal/anaconda3/envs/amitconda/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: The Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW
[[node sequential_1/conv2d_3/Conv2D (defined at /amit/atari-master/game_models/ddqn_game_model.py:130) ]] [Op:__inference_predict_function_427]

The reward is not clip in the codes

if clip: np.sign(reward)
this operation can not change the value of reward, and it should be
if clip: reward = np.sign(reward)

Maybe it's the problem of the NumPy version. In my numpy environment, sign didn't support in-place update.

Got a problem while step_update.

Hi Greg

Thank you for knowledge. I'm studying about reinforcement learning and I try to adept your work on my Kaboom-Atari2600 that based on gym-retro and i got a problem

I try to shape my environment like you. but got a problem in _step_update() I don't know how to fix it

Thank You for your help.

How can I run it on a gpu?

I have write os.environ["CUDA_VISIBLE_DEVICES"] = "0" in atari.py, but it doesn't work, help me plz

What changes could be done for implementing a regular DQN with minor changes??

I am trying to re-use the code for implementing a a regular DQN. So could you guide me what updates needs to be done for the same??

Thanks

Exponential increase in loss

Hi @gsurma,

Thank you for the wonderful code and the medium article. I tried implementing your code but found that the loss function in my model shoots off after some time.

These are the hyper-parameters I used:

initialize environment

env = MainGymWrapper.wrap(gym.make('SpaceInvaders-v0'))
#env = gym.make('SpaceInvaders-v0')

define hyperparameters

total_step_limit = 5000000
wandb.config.episodes = 1000
GAMMA = 0.99
MEMORY_SIZE = 350000
BATCH_SIZE = 32
TRAINING_FREQUENCY = 4
TARGET_NETWORK_UPDATE_FREQUENCY = 40000
MODEL_PERSISTENCE_UPDATE_FREQUENCY = 10000
REPLAY_START_SIZE = 50000
action_size = env.action_space.n
EXPLORATION_MAX = 1.0
EXPLORATION_MIN = 0.1
EXPLORATION_TEST = 0.02
EXPLORATION_STEPS = 425000
EXPLORATION_DECAY = (EXPLORATION_MAX-EXPLORATION_MIN)/EXPLORATION_STEPS
wandb.config.batch_size = 32
wandb.config.learning_rate = 0.00025
input_shape = (4, 84, 84)

The CNN is the same. I also used np.sign for the rewards I got.

Can you guide me on what might be possibly going wrong?

hello why the module not found in the demo code

while i ran python atari.py --help

raise exception like this:

ModuleNotFoundError: No module named 'atari_py'

can't apply this setattr to instance object

@gsurma when i run it without rendering it run's fine but when i enable rendering it gives me this error

Traceback (most recent call last):
File "atari.py", line 100, in
Atari()
File "atari.py", line 20, in init
self._main_loop(self._game_model(game_mode, game_name, env.action_space.n), env, render, total_step_limit, total_run_limit, clip)
File "atari.py", line 45, in _main_loop
env.render()
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/core.py", line 235, in render
return self.env.render(mode, **kwargs)
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/envs/atari/atari_env.py", line 152, in render
from gym.envs.classic_control import rendering
File "/home/overthinker/.local/lib/python2.7/site-packages/gym/envs/classic_control/rendering.py", line 27, in
from pyglet.gl import *
File "/home/overthinker/.local/lib/python2.7/site-packages/pyglet/gl/init.py", line 95, in
from pyglet.gl.lib import GLException
File "/home/overthinker/.local/lib/python2.7/site-packages/pyglet/gl/lib.py", line 149, in
from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
File "/home/overthinker/.local/lib/python2.7/site-packages/pyglet/gl/lib_glx.py", line 38, in
import pyglet.lib
File "/home/overthinker/.local/lib/python2.7/site-packages/pyglet/lib.py", line 55, in
script_path = pyglet.resource.get_script_home()
File "/home/overthinker/.local/lib/python2.7/site-packages/pyglet/init.py", line 337, in getattr
object.setattr(self, '_module', module)
TypeError: can't apply this setattr to instance object

No learning during training

Hi @gsurma,

Thanks for sharing the code, great work there.

I was try to recreate the Breakout model by running the training step. However, even after 7000 runs there seems to be no learning happening. The average score is constant between 1-1.5. However, the accuracy increases and loss decreases.

The code I am using is almost exact clone of the current repo. Could you please let me know if there was any update that was done by you before running the training step for the game?

Tried the same for Pong game and failed to see any learning happening.

Happy to share more details, and any help will be appreciated. Thank you!

Human scores

Hello,

I may have missed information somewhere. Where do average human results come from?

atari_py missing

Hello @gsurma
"atari.py" file is importing atari_py where apparently all the games it can be trained on is listed. Can you please share that file as well?

Plus one more thing, I am trying to run this on my mac, but seems to have some issues with tensorflow. Can you please tell me under what configurations you ran it successfully. (OS, python version etc)

gsurma / atari Goto Github PK

atari's Introduction

Hi there 👋

atari's People

Contributors

Stargazers

Watchers

Forkers

atari's Issues

initialize environment

define hyperparameters

Recommend Projects

Recommend Topics

Recommend Org