ntasfi / pygame-learning-environment Goto Github PK
View Code? Open in Web Editor NEWPyGame Learning Environment (PLE) -- Reinforcement Learning Environment in Python.
License: MIT License
PyGame Learning Environment (PLE) -- Reinforcement Learning Environment in Python.
License: MIT License
It really messed up my research with DQN RL variations, because it is possible to bypass pipe over the screen.
See screenshot: http://www.evernote.com/l/AClrdjOMAFBLSLm1-ZvYqa-63AOtriZ9NoI/
This state gives reward = 1.0 (notice bird partially over pipe)
player height 24, but it is enough to be at position -20 to bypass pipe.
Can you please give me rights to push patch to repo? I am planning to work with your repo and I could find more bugs.
Changes needed:
--- a/ple/games/flappybird/__init__.py
+++ b/ple/games/flappybird/__init__.py
@@ -423,7 +423,7 @@ class FlappyBird(base.PyGameWrapper):
self.lives -= 1
# went above the screen
- if self.player.pos_y < -self.player.height:
+ if self.player.pos_y < 0:
self.lives -= 1
Hi Norm,
Sorry to make this an "Issue", but I did not find any other way of contacting you.
My name is Sven, I am an AI/RL enthusiast with a PhD in Computational Biology and I would like to join forces with you. I just finished the Udacity deep learning nanodegree and am quite fluent in tensorflow, NN-topologies and RL-algos. Also, I have written my own Pygame/RL library (http://www.github.com/sven1977/shine), which is not fully functional yet. It's based on tmx files, through which one can define complete MDPs (games) that can then be solved by different RL algos. It heavily uses the concept of components, which are plug-and-play objects for game agents/enemies/etc.. (e.g. Physics). However, at this point - rather than reinventing the wheel - I would like to contribute to your lib and maybe merge mine (or ideas of it) with PLE. I have also implemented "The Lost Vikings" as a Pygame plugin for my lib (I like the fact that the levels in that game are extremely reward-sparse and would be nice challenges for RL algorithms). Please let me know if you are interested.
Best, Sven
I ran the NaiveAgent in Flappy Bird and noticed that the game runs very fast. Is there a way to slow this down in order to actually see what is happening?
PS: I tried decreasing the FPS, even the allowed_fps attribute of the game, but it did not help
on hit it takes wrong ants position
Here's the code:
import pygame
pygame.init()
clock = pygame.time.Clock()
win = pygame.display.set_mode((1000, 500))
y = 250
class Ant:
def init(self, x, side, colour):
self.x = x
self.width = 20
self.side = side
self.left = False
self.right = True
self.vel = 5
self.counter1 = 0
self.colour = colour
self.dir = dir
self.moving = True
self.counter = 0
def start(self):
if self.side == "Left":
self.right = True
else:
self.left = True
self.counter1 = 1
def side1(self):
if self.x < 500:
return "left"
else:
return "right"
def site(self):
if self.left:
return "left"
else:
return "right"
def move(self):
if self.moving:
if self.counter1 == 0:
self.start()
if self.left:
self.right = False
self.x -= self.vel
else:
self.left = False
self.x += self.vel
def draw(self, win):
self.move()
pygame.draw.circle(win, self.colour, (self.x, 250), 10)
def wall(self):
if self.x + self.width >= 1010:
self.moving = False
elif self.x <= 10:
self.moving = False
def updateWin(win):
win.blit(pygame.display.set_mode((1000, 500)), (0, 0))
text(ant1, ant2, ant3)
ant1.draw(win)
ant2.draw(win)
ant3.draw(win)
pygame.draw.rect(win, (139, 69, 19), (0, 260, 1000, 10))
pygame.display.update()
def hit(ants):
for ant4 in ants:
if ant4.moving:
for ant5 in ants:
if ant4.x < ant5.x:
if ant4.x + ant4.width >= ant5.x:
ant4.right = False
ant4.left = True
ant5.left = False
ant5.right = True
ant4.counter += 1
ant5.counter += 1
def text(ant1, ant2, ant3):
font1 = pygame.font.SysFont("comicsans", 50)
text1 = font1.render(str(ant1.counter), 1, (255, 255, 255))
text2 = font1.render(str(ant2.counter), 1, (255, 255, 255))
text2 = font1.render(str(ant3.counter), 1, (255, 255, 255))
win.blit(text1, (ant1.x, 200))
win.blit(text2, (ant2.x, 200))
win.blit(text1, (ant3.x, 200))
ant1 = Ant(40, "Left", (255, 0, 0))
ant2 = Ant(960, "Right", (100, 255, 255))
ant3 = Ant(200, "Left", (0, 200, 0))
run = True
while run:
clock.tick(15)
keys = pygame.key.get_pressed()
for event in pygame.event.get():
if event.type == pygame.QUIT:
run = False
ants = (ant1, ant2, ant3)
hit(ants)
ant1.site(); ant1.wall()
ant2.site(); ant2.wall()
ant3.site(); ant3.wall()
updateWin(win)
pygame.quit()
I am using the flappy bird game to develop reinforcement learning algorithm. I found that the bird fails at some conditions when it should not fail. So I looked into the code and found that a condition is missing when determining whether the bird hit the tube. Please see my modified code here:
for p in self.pipe_group:
hit = pygame.sprite.spritecollide(
self.player, self.pipe_group, False)
for h in hit:
is_in_pipe = ((p.x - p.width / 2) <= self.player.pos_x < (p.x - p.width / 2))
# do check to see if its within the gap.
top_pipe_check = (
(self.player.pos_y - self.player.height / 2) <= h.gap_start) and is_in_pipe
bot_pipe_check = (
(self.player.pos_y +
self.player.height) > h.gap_start +
self.pipe_gap) and is_in_pipe
I could not see any windows installation, is it suitable for windows?
Thanks
When attempting to use this in combination with gym-ple I notice that the installed PLE does not include assets for some games:
$ sudo python setup.py install
$ cd ../gym-ple
$ python ./gym-ple/example.py
...
pygame.error: Couldn't open /usr/local/lib/python2.7/dist-packages/ple-0.0.1-py2.7.egg/ple/games/flappybird/assets/redbird-upflap.png`
...
$ ls /usr/local/lib/python2.7/dist-packages/ple-0.0.1-py2.7.egg/ple/games/flappybird
__init__.py __init__.pyc
p.act(action)
action takes a value of 119 to push the bird upwards
119 is ASCII number for w which is used in games for upward/forward movement.
but since this game has only two possible action it would be better if it is 0/1
Do you guys know if you can run the dockerfile from a windows host? Not really an issue I guess but not sure where to ask / put this request?
Thanks
from ple import PLE
from ple.games.pong import Pong
game = Pong()
p = PLE(game, fps=30, display_screen=True, force_fps=False)
p.init()
myAgent = MyAgent(p.getActionSet())
nb_frames = 1000
reward = 0.0
for f in range(nb_frames):
if p.game_over(): #check if the game is over
p.reset_game()
obs = p.getScreenRGB()
action = myAgent.pickAction(reward, obs)
reward = p.act(action)
This is my test code.But it did not succeed.The error is:
NameError: name 'MyAgent' is not defined
I did not find MyAgent class in PLE or Pong ,anywhere.So,how can I run the code?
Thanks for all kind humans!
position of the keys in state (where state = p.getGameState() and is a dictionary of key value pairs ) changes every time the game is re run
lets us say that in one instance player_vel is the first key as shown in the figure below
but in another run this changes
how one is supposed to train an reinforcement learning algorithm with this type of observation inputs
MacBook Air --MAC OS 10.14.6
Python3.73 64bit
Wrappered the environment:
`#Code based on https://github.com/KibaAmor/rl-code/blob/master/demos/dqn_flappybird/flappybird_wrapper.py
import numpy as np
from gym import Env, spaces
from ple import PLE
from ple.games import FlappyBird
class FlappyBirdWrapper(Env):
def init(self, **kwargs):
self.game = FlappyBird()
self.p = PLE(self.game, **kwargs)
self.action_set = self.p.getActionSet()
#
self.observation_space = spaces.Discrete(3)
#
self.action_space = spaces.Discrete(2)
def _get_obs(self):
#
state = self.game.getGameState()
#
dist_to_pipe_horz = state["next_pipe_dist_to_player"]
dist_to_pipe_bottom = state["player_y"] - state["next_pipe_top_y"]
velocity = state['player_vel']
return np.array([dist_to_pipe_horz, dist_to_pipe_bottom, velocity])
def reset(self):
self.p.reset_game()
return self._get_obs()
def step(self, action):
reward = self.p.act(self.action_set[action])
obs = self._get_obs()
done = self.p.game_over()
return obs, reward, done, dict()
def seed(self, *args, **kwargs):
pass
def render(self, *args, **kwargs):
pass`
Error happened when running:
File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Test.py", line 31, in <module> watch() File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Test.py", line 19, in watch next_state, reward, done, _ = env.step(action) File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Env.py", line 34, in step reward = self.p.act(self.action_set[action]) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 376, in act return sum(self._oneStepAct(action) for i in range(self.frame_skip)) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 376, in <genexpr> return sum(self._oneStepAct(action) for i in range(self.frame_skip)) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 395, in _oneStepAct self._setAction(action) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 411, in _setAction self.game._setAction(action, self.last_action) File "/Users/Sam/PyGame-Learning-Environment/ple/games/base/pygamewrapper.py", line 79, in _setAction pygame.event.post(ku) pygame.error: posted event keycode must be int
And I found the last action may be '[]', error happens here
if last_action is None: last_action = self.NOOP
if
last_action is None or last_action==[]:` fixed the issue
(py36) ➜ PyGame-Learning-Environment git:(master) sudo pip install -e .
The directory '/home/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Obtaining file:///home/PyGame-Learning-Environment
Requirement already satisfied: Pillow in /usr/lib/python2.7/dist-packages (from ple==0.0.1)
Requirement already satisfied: numpy in /usr/lib/python2.7/dist-packages (from ple==0.0.1)
Installing collected packages: ple
Found existing installation: ple 0.0.1
Can't uninstall 'ple'. No files were found to uninstall.
Running setup.py develop for ple
Successfully installed ple
(py36) PyGame-Learning-Environment git:(master) pip install gym_ple
Collecting gym_ple
Using cached https://files.pythonhosted.org/packages/0d/d6/41e3510bf943a6383b89d1b3db785efbd2c63c55ac2947a4ba31bf9c9009/gym_ple-0.3.tar.gz
Building wheels for collected packages: gym-ple
Running setup.py bdist_wheel for gym-ple ... done
Stored in directory: /home/.cache/pip/wheels/48/a7/28/fd4f7105b5960ce04ddeb161679e2343a02e037ec34e992a54
Successfully built gym-ple
Installing collected packages: gym-ple
Successfully installed gym-ple-0.3
(py36) PyGame-Learning-Environment git:(master) python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ple
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/PyGame-Learning-Environment/ple/__init__.py", line 1, in <module>
from .ple import PLE
File "/home/shekharrajak/Documents/githubRepos/personal/rl_games/tools/PyGame-Learning-Environment/ple/ple.py", line 5, in <module>
import pygame
ModuleNotFoundError: No module named 'pygame'
>>> exit()
(py36) PyGame-Learning-Environment git:(master) pip install pygame
Collecting pygame
Downloading https://files.pythonhosted.org/packages/8e/24/ede6428359f913ed9cd1643dd5533aefeb5a2699cc95bea089de50ead586/pygame-1.9.6-cp36-cp36m-manylinux1_x86_64.whl (11.4MB)
100% |████████████████████████████████| 11.4MB 2.5MB/s
Installing collected packages: pygame
Successfully installed pygame-1.9.6
(py36) PyGame-Learning-Environment git:(master) python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ple
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
couldn't import doomish
Couldn't import doom
>>>
Let me know if I did something wrong.
Hello,
I'm trying out your code on Python 3 and have found a problem in that python 3 disallows mixing tabs and spaces for indentation. As a result the library doesn't work /w 3. For some context around this see:
You can probably change all the files in bulk using sed, like in this example:
from .catcher import Catcher
File "/home/vitob/git_projects/PyGame-Learning-Environment/ple/games/catcher.py", line 3, in
from utils import percent_round_int
ImportError: cannot import name 'percent_round_int'
I can only see vec2d.py in the dir ple/games/utils.
I know PLE has an argument to run without updating the screen, but the screen window is created regardless, which seems odd. I'm trying to run simulations on headless servers, but because of this bug I can't. Is there a work-around that I'm missing?
Hey, can you give a list of agents that we can use and a bit of docs on how to incorporate the agent to this project
Py game turorial
When I run the .lives() function from a Monster Kong game it always returns 0 (on a reset game, on an initialized game, and after getting hit by a fireball). When I trained on Catcher and some of the other games, I ended episodes after a life was lost but couldn't do that for Monster Kong since I couldn't tell how many lives are left. As a work around I modified the starting lives to 1 instead of 3.
Hello,
Do you use pygame or pygame-ce for this library ?
I'm currently trying to port Chrome's T-Rex Game to PyGame Learning Environment but having a hard time of doing it. Can anyone help out?
Hi, how can I manage to run the monsterkong or flappy bird game? They don't have a main method attached to them.
I am able to run all the other games like Pong etc. via python.
Thanks!
The main of raycast.py has a small error, namely that it's not passing any "actions" parameter:
TypeError: __init__() takes exactly 12 arguments (11 given)
which refers to this call:
rc = RayCastPlayer(
map_grid,
init_pos,
init_dir,
width,
height,
resolution,
move_speed,
turn_speed,
plane,
block_types
)
I am trying to use A3C to train flappy bird, however, when I create multiple thread, it crashed. Does any body meet this problem? The error code appears at this line:
p = PLE(game, fps=30, display_screen=True)
p.init()
The error is
[xcb] Unknown sequence number while processing reply
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python: ../../src/xcb_io.c:635: _XReply: Assertion `!xcb_xlib_threads_sequence_lost' failed.
Aborted (core dumped)
This is my code:
import threading
import signal
from ple.games.flappybird import FlappyBird
from ple import PLE
stop_requested = False
def flappy_bird(i):
print 'index:===', i
game = FlappyBird()
p = PLE(game, fps=30, display_screen=True)
p.init()
# actions = p.getActionSet()
# print actions
# reward = 0.0
# for i in range(1000):
# if p.game_over():
# p.reset_game()
# # observation = p.getScreenRGB()
# if i % 20 == 0:
# action = actions[0]
# else:
# action = actions[1]
# print action
# reward = p.act(action)
# if stop_requested == True:
# break
return
def signal_handler(signal_, frame_):
print 'You pressed Ctrl+C !'
stop_requested = True
return
def run():
train_treads = []
for i in range(2):
train_treads.append(threading.Thread(target=flappy_bird, args=(i,)))
signal.signal(signal.SIGINT, signal_handler)
for t in train_treads:
t.start()
print 'Press Ctrl+C to stop'
signal.pause()
print 'Now saving data....'
for t in train_treads:
t.join()
return
if __name__ == '__main__':
run()
from ple.games.flappybird import FlappyBird
from ple import PLE
game = FlappyBird()
p = PLE(game, fps=30, display_screen=True)
agent = myAgentHere(allowed_actions=p.getActionSet())
p.init()
reward = 0.0
for i in range(nb_frames):
if p.game_over():
p.reset_game()
observation = p.getScreenRGB()
action = agent.pickAction(reward, observation)
reward = p.act(action)
On line 7 it say does not work. I don't understand the error. I think the problem it could be it's running on a window pc
# pseudocode of event handler from MonsterKong
if event.key == "up":
player.y -= player.getSpeed() / 2
if not player.checkCollision(ladderGroup): # why?
player.y += player.getSpeed() / 2
The import line
from ple.games.raycast_maze import RaycastMaze
should read
from ple.games.raycastmaze import RaycastMaze
Hey @ntasfi . Thanks for putting the Dockerfile
up for everybody
What I did was
$ wget http://raw.githubusercontent.com/ntasfi/PyGame-Learning-Environment/master/docker/Dockerfile
$ docker build -t prodicus/pygame .
$ docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY prodicus/pygame /bin/bash
I get the following error inside the docker container,
root@9bacfcf2f0a4:/ple/examples# python keras_nonvis.py
Using Theano backend.
No protocol specified
No protocol specified
No protocol specified
Fatal Python error: (pygame parachute) Segmentation Fault
Aborted (core dumped)
root@9bacfcf2f0a4:/ple/examples#
What could be the cause of the problem?
Relevant information
$ docker --version
Docker version 1.11.2, build b9f10c9
$ uname -r
4.4.0-28-generic
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: xenial
OSX 10.11, Python 3.5.2
All required packages are well installed, but still get following error.
Traceback (most recent call last):
File "", line 1, in
File "/Users/Name/workspace/PyGame-Learning-Environment/ple/init.py", line 1, in
from .ple import PLE
File "/Users/Name/workspace/PyGame-Learning-Environment/ple/ple.py", line 5, in
from games import base
when I installed PLE, I want to import ple but got this error
import ple
pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
couldn't import doomish
Couldn't import doom
Wanting to use this alongside openai gym, but it doesn't seem to support python3. After installation, importing ple yields the error:
"No module named 'games'"
I'm assuming this is due to the import differences between the two versions; is this a known issue? Is python3 support planned?
I am referring to there two lines here
Could you explain why did you clone your repo twice here?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.