kandouss / marlgrid Goto Github PK

View Code? Open in Web Editor NEW

137.0 5.0 25.0 173 KB

Gridworld for MARL experiments

License: Apache License 2.0

Python 100.00%

marlgrid's Introduction

MarlGrid

Gridworld for MARL experiments, based on MiniGrid.

Training multiple independent learners

Pre-built environment

MarlGrid comes with a few pre-built environments (see marlgrid/envs):

MarlGrid-3AgentCluttered11x11-v0
MarlGrid-3AgentCluttered15x15-v0
MarlGrid-2AgentEmpty9x9-v0
MarlGrid-3AgentEmpty9x9-v0
MarlGrid-4AgentEmpty9x9-v0 (as of v0.0.2)

Custom environment

Create an RL agent (e.g. TestRLAgent subclassing marlgrid.agents.LearningAgent) that implements:

action_step(self, obs),
save_step(self, *transition_values),
start_episode(self) (optional),
end_episode(self) (optional),

Then multiple such agents can be trained in a MARLGrid environment like ClutteredMultiGrid:

agents = marlgrid.agents.IndependentLearners(
    TestRLAgent(),
    TestRLAgent(),
    TestRLAgent()
)

env = ClutteredMultiGrid(agents, grid_size=15, n_clutter=10)


for i_episode in range(N_episodes):

    obs_array = env.reset()

    with agents.episode():

        episode_over = False

        while not episode_over:
            # env.render()

            # Get an array with actions for each agent.
            action_array = agents.action_step(obs_array)

            # Step the multi-agent environment
            next_obs_array, reward_array, done, _ = env.step(action_array)

            # Save the transition data to replay buffers, if necessary
            agents.save_step(obs_array, action_array, next_obs_array, reward_array, done)

            obs_array = next_obs_array

            episode_over = done
            # or if "done" is per-agent:
            episode_over = all(done) # or any(done)

marlgrid's People

Contributors

Stargazers

Watchers

marlgrid's Issues

"gl" not defined error (probably resolved but need verification)

in "marlgrid/utils"; the rendering.py file has errors.

gl.glTexParameteri(gl.GL_TEXTURE_2D,
            gl.GL_TEXTURE_MAG_FILTER, gl.GL_NEAREST)

Following are the imports from original repo:-

import pyglet
from pyglet.gl import *
import sys

and I updated it:-

import pyglet
from pyglet.gl import *
import sys
from OpenGL.GL import *
from OpenGL.GLU import *
import OpenGL.GL as gl

this resolved the problem (just no errors as of now) ig. But still I dont know as I have not been able to run the project.
i guess it needed pyglet to be installed

Addition of multi-agent environments

Hello! Thank you for making this extremely useful extension to minigrid. Do you plan to add more environments from literature (e.g. predator-prey) and make this repository as robust as gym-minigrid? If so will you have contribution guidelines in the future for pull requests?

Missing class(es) from readme

The readme recommends creating an RL agent that subclasses "marlgrid.agents.LearningAgent", which does not exist in the code. Should this be objects.GridAgent?

Likewise, the example code references agents.IndependentLearners, which is not in the repo. I am unsure how that should be structured.

I am relatively new to gym environments, so please forgive me for misunderstanding something that may be obvious.

TypeError: init() missing 1 required positional argument: 'save_root

In examples folder, while running video_test.py throws the following error.

abhishek@abhishek-ubuntu:~/Desktop/marlgrid-master$ /usr/bin/python3 /home/abhishek/Desktop/marlgrid-master/examples/video_test.py

/usr/local/lib/python3.8/dist-packages/gym_minigrid-1.0.2-py3.8.egg/gym_minigrid/roomgrid.py:302: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if front_cell is None or front_cell.type is 'wall':
Traceback (most recent call last):
  File "/home/abhishek/Desktop/marlgrid-master/examples/video_test.py", line 7, in <module>
    env = GridRecorder(env, render_kwargs={"tile_size": 11})
TypeError: __init__() missing 1 required positional argument: 'save_root'

Problem solved using pycharm

I copied the steps from your video:

pip install git+git://github.com/kandouss/marlgrid.git

import gym, marlgrid.envs
env = gym.make('Goalcycle-demo-solo-v0')

Now, when I am trying to run the "video_test.py"(this file did not get downloaded using the first command), it throws the error.

Observation shapes do not match

env = gym.make('MarlGrid-2AgentEmpty9x9-v0')

act = env.action_space.sample()
obs, rew, done, _ = env.step(act)
obs = np.array(obs)

obs.shape gives me (2, 56, 56, 3), while env.observation_shape gives me Tuple(Box(7, 7, 3), Box(7, 7, 3)).
Mismatch between the two.

Interest in contributing upstream

Hi @kandouss, great work here! Thank you :)
I was glad to find your repo when I was about to implement the same stuff from scratch.

I've been running some tests with marlgrid and therefore have fixed some compat. issues with newer numpy version as well some other minor bugs I've found as I tried to implement my own examples.

I'm wondering if you're interested in me to contribute these fixes upstream?
I'm also thinking about adding more examples and docs to the repo to make it more accessible.

Best!