Error When running ddpg Example. about gym-unrealcv HOT 8 CLOSED

Actually, for those who also have the same problem, I solve this by adding env = env.unwrapped.

Then I have the assertion error so I changed 'Search-RrDoorDiscrete-v0' to 'Search-RrDoorContinuous-v0'

After that, it complained that it can not find env.observation_space then I added env.observation_space = env.observation_shape in the run.py file.

I am not sure if it is correct, but it now runs.

from gym-unrealcv.

Hi,
Thank you for your report.
Can you report your gym version? The older version gym may not need env.unwrapped.
Changing the Discrete to Continuous in env name is necessary to any gym version when using DDPG for continuous actions.

from gym-unrealcv.

Hi,

Thanks a lot for your consideration and the reply! My gym version is 0.9.3. I also think it is the problem of gym version. The older versions should work.

I have tried to run the example code of the DDPG algorithm after the previous steps. However, the agent did not learn after the training (Judging from the rewards from the last 30 episodes). I am not sure what might be the problem in this case.

btw, I followed the similar procedure for the DQN example provided in this repo and the agent learned.

Best,

from gym-unrealcv.

Hi,
Could you tell me the env and the type of reward function you used when training the DDPG? Usually, DDPG needs more exploration and dense reward to train.

from gym-unrealcv.

Hi, I tried both bbox and bbox_distance reward functions.

from gym-unrealcv.

Hi,

Thanks for the reply.

I used the example you provided. The modifications I had is to change ENV_NAME in the constants.py to be 'Search-RrDoorContinuous-v0' and change the reward type to be 'bbox_distance'.

Here is the configuration string I had.

register(
    id='Search-RrDoorContinuous-v0',
    entry_point='gym_unrealcv.envs:UnrealCvSearch_base',
    kwargs = {'setting_file' : 'search_rr_door41.json',
              'reset_type' : 'waypoint',
              'test': False,
              'action_type' : 'continuous',
              'observation_type': 'color',
              'reward_type': 'bbox_distance',
              'docker': use_docker
              },
    max_episode_steps = 1000000
)

from gym-unrealcv.

Hi,

The environment seems no problem.
You can try to modify the hyperparameter in the constants.py, especially changing MAX_EXPLORE_STEPS to 50000 or larger for more exploration.

from gym-unrealcv.

Thanks a lot! I will have a try.

from gym-unrealcv.