How to use uniform control policy? about rllab HOT 8 OPEN

Have you tried this one? https://github.com/openai/rllab/blob/master/rllab/algos/nop.py

from rllab.

I tried NOP got the following error:

Traceback (most recent call last):
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 115, in
run_experiment(sys.argv)
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 102, in run_experiment
maybe_iter = concretize(data)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1018, in concretize
return method(_args, *_kwargs)
File "/home/data/lchenat/rllab-master/rllab/algos/batch_polopt.py", line 250, in train
paths = self.sampler.obtain_samples(itr)
File "/home/data/lchenat/rllab-master/rllab/algos/batch_polopt.py", line 27, in obtain_samples
cur_params = self.algo.policy.get_param_values()
File "/home/data/lchenat/rllab-master/rllab/core/parameterized.py", line 57, in get_param_values
for param in self.get_params(**tags)]
File "/home/data/lchenat/rllab-master/rllab/core/parameterized.py", line 34, in get_params
if tag_tuple not in self._cached_params:
AttributeError: 'UniformControlPolicy' object has no attribute '_cached_params'

I am a bit confused about this, because in parameterized.py _cached_params is actually set:

class Parameterized(object):

def __init__(self):
    self._cached_params = {}
    self._cached_param_dtypes = {}
    self._cached_param_shapes = {}

from rllab.

Hmm weird. I cannot reproduce the error on my end. What if in the initializer of UniformControlPolicy, you explicitly invoke Parameterized.init(self)?

from rllab.

i invoke Parameterized.init(self) in Policy, it seems that I have an old version of RLLAB. I then got this error:

Traceback (most recent call last):
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 115, in
run_experiment(sys.argv)
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 102, in run_experiment
maybe_iter = concretize(data)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1018, in concretize
return method(_args, *_kwargs)
File "/home/data/lchenat/rllab-master/rllab/algos/batch_polopt.py", line 251, in train
samples_data = self.sampler.process_samples(itr, paths)
File "/home/data/lchenat/rllab-master/rllab/algos/batch_polopt.py", line 73, in process_samples
ent = np.mean(self.algo.policy.distribution.entropy(agent_infos))
AttributeError: 'UniformControlPolicy' object has no attribute 'distribution'

from rllab.

I am using python 2.7, and I got the same problem after downloading the rllab-py2 again.

from rllab.

By the way, when I am running multiple python scripts for different algorithms, I got the following errors:

using seed 8
using seed 8
Could not import matplotlib.pyplot, therefore cma.plot()" + etc. is not available
Error while instantiating <class 'rllab.policies.gaussian_mlp_policy.GaussianMLPPolicy'>
Traceback (most recent call last):
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1032, in concretize
_args, *_kwargs)
File "/home/data/lchenat/rllab-master/rllab/policies/gaussian_mlp_policy.py", line 110, in init
super(GaussianMLPPolicy, self).init(env_spec)
File "/home/data/lchenat/rllab-master/rllab/policies/base.py", line 7, in init
super(Policy, self).init()
TypeError: init() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 115, in
run_experiment(sys.argv)
File "/home/data/lchenat/rllab-master/scripts/run_experiment_lite.py", line 102, in run_experiment
maybe_iter = concretize(data)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1014, in concretize
obj = concretize(maybe_stub.obj)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1029, in concretize
kwargs = concretize(maybe_stub.kwargs)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1044, in concretize
ret[concretize(k)] = concretize(v)
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 1038, in concretize
ret = maybe_stub.stub_cache
File "/home/data/lchenat/rllab-master/rllab/misc/instrument.py", line 155, in __getattr
raise AttributeError('Cannot get attribute %s from %s' % (item, self.proxy_class))
AttributeError: Cannot get attribute __stub_cache from <class 'rllab.policies.gaussian_mlp_policy.GaussianMLPPolicy'>

Is it ok to run several algorithms in different scripts at the same time?

edit: oh, it is not the problem of running several scripts, it is actually caused by adding "super(Policy, self).init()" in init of policy base class. I added this because it seemed to solve the uniform control policy problem.

from rllab.

I just pushed a fix: 55b7df1 to the attribute error issue.

I recommend upgrading to the master branch which uses python 3. Also, it is recommended to explicitly call the superclass constructor instead of using super.

from rllab.

done! Thanks for your update.

from rllab.