Errors about training with Soft-body about maniskill HOT 8 CLOSED

Oh, I had mistaked the buffer_filenames path, now checking the submission...

from maniskill.

I may have encountered the same issue as before under env_name=OpenCabinetDoor-v1. I am able to have a score outside of the Docker image locally, but when running within the Docker image (using the latest version of ManiSkill2), the score is 0. The submission ID for this issue is 647866f9e09b0d48eff26846. Is it possible that the cause of this issue is:

After private discussion with the original poster, the conclusion is as follows so far

The scores on the leaderboard are indeed correct and the leaderboard is evaluating submissions correctly. The problem lies in the user_solution.py file which leverages ManiSkill2-Learn. Currently OpenCabinetDoor and the other ManiSkill1 environments do not return agent base_pose values to the observation. To address that the ManiSkill2-Learn framework automatically adds it by using the actual environment's robot information.

However, in submissions, the user_solution.py file creates a dummy environment to process observations which includes adding the base_pose. This dummy environment is not the actual environment so there is a mismatch in results as running the ManiSkill2-Learn evaluation will produce successes but running the actual evaluation where there is no access to the actual environment will cause errors as the base_pose values are wrong (and this is also why the robot looks like its only going one direction, the observation is not really changing)

We are currently discussing fixes for this.

from maniskill.

I'll verify today/tomorrow, but it's not immediately clear at the moment

from maniskill.

Hi, has the stage 2 of evaluation begun?

from maniskill.

Is June 3rd, 2023 the deadline?

from maniskill.

from maniskill.

OK，but my submission has stuck for about 13 hours, what's the problem?

from maniskill.

Communicated and resolved on 6/5.

from maniskill.