How to modify code for continuous actions? about pytorch-a3c HOT 5 CLOSED

Got nearly working code in this thread,

https://discuss.pytorch.org/t/continuous-action-a3c/1033

from pytorch-a3c.

Hi, any chance you could give me some advice? I'm still stuck trying to get this to work? Here's a post of my code

https://gist.github.com/AjayTalati/184fec867380f6fa22b9aa0951143dec

I keep getting this error,

File "main_single.py", line 174, in <module>
value_loss = value_loss + advantage.pow(2)
AttributeError: 'numpy.ndarray' object has no attribute 'pow'

I don't understand why advantage has become a numpy array instead of a torch.tensor - it never occurred with the discrete action implementation?

Any ideas what I've got wrong?

Thanks a lot for your help,

Best,

Ajay

from pytorch-a3c.

Closing this, as continuous functions are just a pain to approximate?

from pytorch-a3c.

I will add continuous control later. I don't have time at the moment.

from pytorch-a3c.

OK - cool - take your time 👍 I don't mean this as an A3C specific comment, or anything specific about your implementation.

It's just a general observation, (and perhaps a provable fact), that I've found discrete functions easier to approximate that continuous ones.

In terms of simple MLP theory, this is nice by Mhaskar and Poggio,

Learning Real and Boolean Functions: When Is Deep Better Than Shallow

from pytorch-a3c.