I have employed macpo on an environment I have developed myself on resources allocation in 5g, but I am getting errors as follows.
main(sys.argv[1:])
File "/home/mzi/ran-slicing-simulation/scripts/train/train_macpo.py", line 163, in main
runner.run()
File "/home/mzi/ran-slicing-simulation/scripts/../runners/slicing_runner_macpo.py", line 75, in run
train_infos = self.train()
File "/home/mzi/ran-slicing-simulation/scripts/../runners/base_runner_macpo.py", line 171, in train
train_info = self.trainer[agent_id].train(self.buffer[agent_id])
File "/home/mzi/ran-slicing-simulation/scripts/../algorithms/r_macpo.py", line 629, in train
= self.trpo_update(sample, update_actor)
File "/home/mzi/ran-slicing-simulation/scripts/../algorithms/r_macpo.py", line 385, in trpo_update
g_step_dir = self.conjugate_gradient(self.policy.actor,
File "/home/mzi/ran-slicing-simulation/scripts/../algorithms/r_macpo.py", line 227, in conjugate_gradient
_Avp = self.fisher_vector_product(actor, obs, rnn_states, action, masks, available_actions, active_masks, p)
File "/home/mzi/ran-slicing-simulation/scripts/../algorithms/r_macpo.py", line 241, in fisher_vector_product
kl = self.kl_divergence(obs, rnn_states, action, masks, available_actions, active_masks, new_actor=actor,
File "/home/mzi/ran-slicing-simulation/scripts/../algorithms/r_macpo.py", line 215, in kl_divergence
return kl.sum(1, keepdim=True)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
I tried to find the root cause behind the error, and it seems something wrong with action_logits properties: it cannot calculate mean or std deviation, although action log probabilities are fine. Find the screen shot attached below:
I believe this goes back to my action space being Categorical as std and mean does not make sense in such a scenario.
I was wondering how I can exploit macpo for my case.