Mamujoco v4 about marllib HOT 7 CLOSED

We have recently updated MAMuJoCo to its latest version, which is now maintained under Gymnasium-Robotics.

To learn how to use it, you can refer to the examples provided at Examples.

Please note that Gymnasium-Robotics has not been merged into the master branch. You can access it by checking out the rllib_1.8.0_dev branch.

If you encounter any bugs or issues, we encourage you to create a pull request (PR). We welcome contributions from the community.

from marllib.

I think so. You can pip show mujoco to see the MuJoCo version you are using and then check the corresponding half_cheetah version you are using.
Additionally, it's worth mentioning that we haven't conducted any finetuning on gymnasium_mamujoco, so you will need to perform the finetuning yourself.

from marllib.

There are two approaches to finding optimal hyperparameters:

1. Referencing benchmarking papers

2. Using grid search with ray.tune.grid_search()

Both of these approaches have their advantages. Consulting benchmarking papers provides insights based on prior research, while grid search systematically explores the hyperparameter space.

MARLib's fine tuning is mostly based on 1.

from marllib.

Hi

Thank you so much.

Is the below code the correct way to use half_cheetah_v4 (version 4)?

from marllib import marl
env = marl.make_env(environment_name="gymnasium_mamujoco", map_name="2AgentHalfCheetah", force_coop=True)
mappo = marl.algos.mappo(hyperparam_source="mamujoco")
model = marl.build_model(env, mappo, {"core_arch": "mlp", "encode_layer": "128-256"})
mappo.fit(env, model, stop={'episode_reward_mean': 2000, 'timesteps_total': 20000000}, local_mode=False, num_gpus=1,
num_workers=60, share_policy='group', checkpoint_freq=500)

from marllib.

Similarly you mentioned that Sisl is not finetuned. Can you please let me know how exactly fine tuning is done as I am a beginner in Marl. Also for mpe (simple spread) and mamujoco (v2) with your finetuned parameters, the reward was increasing from a lower value to a higher value. Like for spread it went from -120 to -65 and then plateaus. Similarly for mamujoco v2 it starts from -600 and goes to 2000 and then run stops due to 2000 reward stopping criteria.

While for waterworld I observed it starts with reward of -6 and then increases for few iterations and then drops to -5 and stays there. Similar behaviour I observed for multiwalker (starts from -300, increases for few iterations and then drops to -300 and then plateaus) and gymnasium mamujoco v4 (-600 to -1300 and then plateaus). These are the observations with MAPPO algorithm. Is it because they are not finetuned? Will a finetuned environment always have an increasing reward? In a paper, I saw a plot for multiwalker with PPO following exactly same trend that I mentioned. Also in some of your results the reward drops from original value and plateaus. So what exactly is finetuning and how to do it? Simply put, in what ways will a finetuned environment differ from a non-finetuned environment for a given algorithm.

Also, how do I ensure that whatever reward pattern I have observed for an environment is correct? Are there any standard reward patterns for an environment with a given algorithm? How do I select the number of timesteps? It's generally taken around 1e7. Is it taken to ensure that reward plateaus?

Thank you.

from marllib.

Only tasks show up in this directory have been finetuned.
For other questions, it is challenging to address them comprehensively within a single issue reply. We encourage you to explore related papers and surveys. They can provide a more in-depth understanding of the broader concepts and help you find the answers you seek: link.

from marllib.

Can you please let me know how exactly you finetuned?

from marllib.