visualise_behaviour() and evaluate() are used in the log() function respectively
visualise_behaviour() function uses test_env, but the results are not written to the log and are only printed out
evaluate() function uses train_env, the result is written in the log
I plot the result diagram of the two functions and find that the result of evaluate() function is close to that of the paper, but the result of visualise_behaviour() function is poor
This is the result of three seeds in the environment of half_cheetah_vel

I‘m so sorry to bother you again. In my own env, I find the action will extremely beyond the action limitation. Then I check the policy act function, there is no limit for the output action which is sampled from the FixNormal. Should I add the action.tanh() in the code?
Here is the policy action I recorded. . My action limit is [-1,1]. But the action in your env (hf vel) seems to work well. Though the action sometimes will beyond the -1 or 1 too.

I'm sorry to bother you. When I try your code, the 48g GPU Memory is still not enough. And I just try 2 works and 1 mixture work. I wonder how I sove the problem.