ymzhang01 / focops Goto Github PK

I tried this on Safety gym with just changing env_name, using info['cost'] for constraint calculations, and computing cost returns as undiscounted sum of costs instead of discounted sum of cost -
`ret_eps += rew

cost_ret_eps += cost`
(SInce Safety Gym aim is to max sum of rewards and also constrain undiscounted sum of costs where max episode length is 1000 steps)

Performance is quite poor on even Point Goal level 2 environment with costs not getting constrained at all.
What can be reason for that?
https://openai.com/blog/safety-gym/

Hi,
I am reproducing the code on the example you given. (Humanoid task in the robots with speed limits experiments (using the default parameters)).
However, the performance is really bad. I have compared the hyper-parameters with the list in the paper. I am wondering if I missed something important? What can be reason for that?

Hello! I have read your FOCOPS paper and would like to reproduce the results. I found that in MuJoCo environments the lengths of different episodes may differ largely, ranging from 20- to 2000+. Thus, the cost attributed to each step may be hard to adjust by the policy if you fix the total safety budget for episodes with different lengths, and I found that the naive sac-lag algorithm performs poorly under the same constraint with FOCOPS, in the sense that when the episode step becomes large, the total cost would also become large.
How do you fix this problem?
(BTW, I can't found the evaluation part in the focops code, did you fix the episode step during training or evaluating?)