Giter VIP home page Giter VIP logo

scalable_maddpg's Introduction

scalable_maddpg

scalable multi agent reinforcement learning. Details can be found in the Report

to do list

  • tune L2, does LSTM parameters need L2 regulizer?
  • fix environments
  • fix rewards
  • decrease the frequency of summaries
  • rearrange main.py
  • prey boundary problem
  • modify the initial position of the agents and prey
  • add another network for prey
  • add summary for rewards of each episode

Alternative to Gym

An alternative of Gym environment is created (env.py). The rendering implementation of the envrionment is matplot. So, it would be much easier to use. However, you need to implement the prey policy by yourself.

Results

Here, we have done two independent runs. In each run, from episode 1 to episodes 3x10^4, three agents were in the game. At episode 3x10^4, we added three more agents into this game. Here we show the mean Q value of all the agents in our experiments.

Demo results

In this demon, the prey walks randomly. Agents learn to catch the prey.

scalable_maddpg's People

Contributors

livey avatar zsy93 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

scalable_maddpg's Issues

i run main.py,the error:

Traceback (most recent call last):
File "/home/shy/桌面/scalable_maddpg-master/main.py", line 6, in
from env import envs
ImportError: cannot import name 'envs'

Module not registered

When I try to run the code using main.py I got the this error:

  File "/home/USER/myprojects/scalable_maddpg/env/multiagent/__init__.py", line 11, in <module>
    max_episode_steps=100,
  File "/home/USER/.local/lib/python2.7/site-packages/gym/envs/registration.py", line 161, in register
    return registry.register(id, **kwargs)
  File "/home/USER/.local/lib/python2.7/site-packages/gym/envs/registration.py", line 154, in register
    raise error.Error('Cannot re-register id: {}'.format(id))
gym.error.Error: Cannot re-register id: MultiagentSimple-v0

I tried to install the enviroment provided, but it didn't work. Do you have any suggestions?

TypeError: tuple indices must be integers or slices, not tuple

Hello.

Thanks for your codes, but I got some errors.

I’m using your env.py, so I renamed the env folder and changed from env import envs to import env as envs.
Is it correct?

Since I only have CPU, I change self.sess = tf.Session(config=tf.ConfigProto(log_device_placement=True)) to
self.sess = tf.Session()
But maybe this is not related to the errors.

When I ran the codes, I get such errors:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/helpers/pydev/pydev_run_in_console.py", line 53, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File “xx/scalable_maddpg/main.py", line 44, in <module>
    action = maddpg.noise_action(current_state)
  File “xx/scalable_maddpg/maddpg.py", line 158, in noise_action
    action = self.action(state)
  File “xx/scalable_maddpg/maddpg.py", line 138, in action
    action[ii,:] = self.agents[ii].action(state[ii,:])
TypeError: tuple indices must be integers or slices, not tuple

I tried to solve it by changing action[ii,:] = self.agents[ii].action(state[ii,:]) to action[ii,:] = self.agents[ii].action(state[ii]), but it lead to other errors.

Do you have some idea of the errors?
Thank you.

TypeError: 'int' object is not iterable

Hello!
I have deleted the env.py from the main dictionary, but come into this error when running the main.py. Do you have any idea about this issue? Thanks a lot!
File "E:/maddpg/scalable_maddpg-master/main.py", line 16, in
Env = envs.Environ(num_agents,max_edge)
File "E:\maddpg\scalable_maddpg-master\env\envs.py", line 23, in init
= self.create_env(num_agents)
File "E:\maddpg\scalable_maddpg-master\env\envs.py", line 37, in create_env
shared_viewer=False)
File "env\multiagent\environment.py", line 71, in init
self.observation_space.append(spaces.Box(low=-np.inf, high=+np.inf, shape=(obs_dim),))
File "C:\Users\Administrator\Anaconda3\lib\site-packages\gym\spaces\box.py", line 34, in init
gym.Space.init(self, shape, dtype)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\gym\core.py", line 202, in init
self.shape = None if shape is None else tuple(shape)
TypeError: 'int' object is not iterable

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.