How to run the train_language_agent.py without using slurm about grounding_llms_with_online_rl HOT 2 OPEN

and I run the example/ppo_finetuning for BabyAI-MixedTrainLocal enviroment in lamorel with modified config

rl_script_args:
  path: ???
  name_environment: 'BabyAI-MixedTrainLocal'

  #'BabyAI-GoToRedBall-v0'
  #'BabyAI-MixedTrainLocal'
  #'BabyAI-GoToRedBall-v0'
  #'BabyAI-MixedTestLocal'
  #'BabyAI-GoToRedBall-v0'
  epochs: 1000
  steps_per_epoch: 1500
  minibatch_size: 64
  gradient_batch_size: 16
  ppo_epochs: 4
  lam: 0.99
  gamma: 0.99
  target_kl: 0.01
  max_ep_len: 1000
  lr: 1e-4
  entropy_coef: 0.01
  value_loss_coef: 0.5
  clip_eps: 0.2
  max_grad_norm: 0.5
  save_freq: 100
  output_dir: "/home/yanxue/lamoral/pposmalltrain"

But it seems failed to train, it only gets score around 0.2 less than 0.6 in your paper

from grounding_llms_with_online_rl.

Hi,

Concerning your first issue, the stack trace you provided misses the real issue that happened so I can't tell. Anyway accelerate has some difficulties with launching two processes on a single machine with only 1 GPU. This is why we provided a custom version of accelerate (which is outdated). Could you try these two PRs (1, 2) please? Or manually launch the two processes as shown in Lamorel's documentation.

Concerning your second issue, this is weird. Let me try to launch some experiments and find out what happens.

from grounding_llms_with_online_rl.