Comments (6)
It seems that you are using the IMPALA algorithm with the customized agent. Could you paste the code of MPEAgent here?
from parl.
right, I'm trying to use multi-agent env.
def __init__(self, algorithm, obs_shape, act_dim,
learn_data_provider=None):
assert isinstance(obs_shape, (list, tuple))
self.obs_shape = obs_shape
self.act_dim = act_dim
# self.place = fluid.CUDAPlace(
# 0) if machine_info.is_gpu_available() else fluid.CPUPlace()
# self.fluid_executor = fluid.Executor(self.place)
super(MPEAgent, self).__init__(algorithm)
if learn_data_provider:
self.learn_reader.decorate_tensor_provider(learn_data_provider)
self.learn_reader.start()
def build_program(self):
self.sample_program = fluid.Program()
self.predict_program = fluid.Program()
self.learn_program = fluid.Program()
# fluid.layers.data()用来接收数据,类似placeholder
with fluid.program_guard(self.sample_program):
obs = layers.data(
name='obs', shape=self.obs_shape, dtype='float32')
self.sample_actions, self.behaviour_logits = self.alg.sample(obs) # sample()
# predict()
with fluid.program_guard(self.predict_program):
obs = layers.data(
name='obs', shape=self.obs_shape, dtype='float32')
self.predict_actions = self.alg.predict(obs)
with fluid.program_guard(self.learn_program):
obs = layers.data(
name='obs', shape=self.obs_shape, dtype='float32')
obs_act = layers.data(
name='obs_act', shape=(-1, 21), dtype='float32')
actions = layers.data(
name='actions', shape=[], dtype='int64')
behaviour_logits = layers.data(
name='behaviour_logits', shape=[self.act_dim], dtype='float32')
rewards = layers.data(
name='rewards', shape=[], dtype='float32')
dones = layers.data(
name='dones', shape=[], dtype='float32')
lr = layers.data(
name='lr', shape=[1], dtype='float32', append_batch_size=False)
entropy_coeff = layers.data(
name='entropy_coeff', shape=[], dtype='float32')
self.learn_reader = fluid.layers.create_py_reader_by_data(
capacity=32,
feed_list=[
obs, obs_act, actions, behaviour_logits, rewards, dones, lr, entropy_coeff
])
obs, obs_act, actions, behaviour_logits, rewards, dones, lr, entropy_coeff = fluid.layers.read_file(
self.learn_reader)
vtrace_loss, kl = self.alg.learn(obs, obs_act, actions, behaviour_logits,
rewards, dones, lr, entropy_coeff)
self.learn_outputs = [
vtrace_loss.total_loss, vtrace_loss.pi_loss,
vtrace_loss.vf_loss, vtrace_loss.entropy, kl
]
self.learn_program = parl.compile(self.learn_program,
vtrace_loss.total_loss)
def sample(self, obs_np):
obs_np = obs_np.astype('float32')
self.fluid_executor.run(fluid.default_startup_program())
# FIXME: error
sample_actions, behaviour_logits = self.fluid_executor.run(
self.sample_program,
feed={'obs': obs_np},
fetch_list=[self.sample_actions, self.behaviour_logits])
return sample_actions, behaviour_logits
def predict(self, obs_np):
obs_np = obs_np.astype('float32')
# self.fluid_executor.run(fluid.default_startup_program())
predict_actions = self.fluid_executor.run(
self.predict_program,
feed={'obs': obs_np},
fetch_list=[self.predict_actions])[0]
return predict_actions
def learn(self):
# self.fluid_executor.run(fluid.default_startup_program())
total_loss, pi_loss, vf_loss, entropy, kl = self.fluid_executor.run(
self.learn_program, fetch_list=self.learn_outputs)
return total_loss, pi_loss, vf_loss, entropy, kl```
from parl.
Thanks for your quick reply! We will try to reproduce the problem at our environment and then fix it.
from parl.
I tried to restore model.ckpt in the default IMPALA alg.The same Error happend.
agent.restore('./model.ckpt')
File "/home/tianqi/anaconda3/lib/python3.6/site-packages/parl/core/fluid/agent.py", line 221, in restore
filename=filename)
File "/home/tianqi/anaconda3/lib/python3.6/site-packages/paddle/fluid/io.py", line 798, in load_params
filename=filename)
File "/home/tianqi/anaconda3/lib/python3.6/site-packages/paddle/fluid/io.py", line 675, in load_vars
raise TypeError("program's type should be Program")
TypeError: program's type should be Program
from parl.
We find that this line causes the issue:
PARL/examples/IMPALA/atari_agent.py
Line 79 in c5a8c2b
We will fix this problem next week:)
Currently, we suggest removing the line in your code. This line transfers a vanilla program into a new program that runs parallel on CPUs. It has little negative affect on the performance if you have a GPU.
from parl.
The issue has been addressed by #192
Please update parl with the following command:
pip install --upgrade git+https://github.com/PaddlePaddle/PARL.git
,
or just download the repository and install it locally with: cd PARL; pip install .
Thanks for your feedback on PARL. It does make PARL a better framework!
from parl.
Related Issues (20)
- pip安装parl时报错
- 电气研究生跨考控制之我真是小白!!! HOT 1
- import parl时RuntimeError HOT 2
- 询问gym库中OBS对象属性的问题
- 使用python train.py运行tutorials中的代码没有反应
- 渲染图像render=True时,代码报错,图像框一下子闪退 Windows10 HOT 1
- torch的选择问题? HOT 2
- DDPG mujoco error
- It seems `fluid` is absent from paddlepaddle after 2.5, causing issue with backend detection on Windows HOT 1
- Attributerror HOT 6
- pip安装问题 HOT 4
- 分发本地文件如何同步更新呢 HOT 1
- Remaining issues of CI HOT 1
- pettingzoo.utils.deprecated_module.DeprecatedEnv: simple_speaker_listener_v3 is now deprecated, use simple_speaker_listener_v4 instead
- WRN No deep learning framework was found, but it's ok for parallel computation. HOT 1
- ValueError: optimizer got an empty parameter list HOT 1
- AttributeError: module 'parl' has no attribute 'Model' HOT 1
- MPC HOT 1
- IMPALA for continous action domains
- 并行计算时一个job从日志看只被分配了一个cpu核心,如何自定义cpu资源的分配? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parl.