ymzhang01 / focops Goto Github PK
View Code? Open in Web Editor NEWPytorch Implementation for First Order Constrained Optimization in Policy Space (FOCOPS).
Pytorch Implementation for First Order Constrained Optimization in Policy Space (FOCOPS).
I tried this on Safety gym with just changing env_name, using info['cost'] for constraint calculations, and computing cost returns as undiscounted sum of costs instead of discounted sum of cost -
`ret_eps += rew
cost_ret_eps += cost`
(SInce Safety Gym aim is to max sum of rewards and also constrain undiscounted sum of costs where max episode length is 1000 steps)
Performance is quite poor on even Point Goal level 2 environment with costs not getting constrained at all.
What can be reason for that?
https://openai.com/blog/safety-gym/
Hi,
I am reproducing the code on the example you given. (Humanoid task in the robots with speed limits experiments (using the default parameters)).
However, the performance is really bad. I have compared the hyper-parameters with the list in the paper. I am wondering if I missed something important? What can be reason for that?
Hello! I have read your FOCOPS paper and would like to reproduce the results. I found that in MuJoCo environments the lengths of different episodes may differ largely, ranging from 20- to 2000+. Thus, the cost attributed to each step may be hard to adjust by the policy if you fix the total safety budget for episodes with different lengths, and I found that the naive sac-lag algorithm performs poorly under the same constraint with FOCOPS, in the sense that when the episode step becomes large, the total cost would also become large.
How do you fix this problem?
(BTW, I can't found the evaluation part in the focops code, did you fix the episode step during training or evaluating?)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.