The evaluation results are different about grip HOT 4 CLOSED

Hi @Bigtuo, thank you for trying our code.
If this is what you get after submitting your results to ApolloScape website, there is a high probability that your code has bugs. Because you did you provide any useful information about how you get this result (Code modification, GPUs, OS, Libraries, etc.), I cannot figure out the reason.

There are so many researchers have tried our code and got duplicated results.
The code we released can get similar results (after submitting results to ApolloScape):
GRIP 1.2524 2.2081 0.7142 1.8024 2.3440 3.9805 1.3732 3.4155

If you are sure that your code does not have any bugs, there are some factors may impact the precision (but not in a large scale):

1. The GPUs. Different GPUs have different precision.
2. Driver/Library version. The NVIDIA driver, CUDA, CUDNN, Pytorch, Python, even OS can impact the precision.

from grip.

I didn't make any changes,and I use APOLLO's evaluation.py. In addition to your inconsistent results, the official baseline evaluation results are also inconsistent.
The evaluation results of baseline model are as follows:
WSADE: 35.52371758873721
ADEv, ADEp, ADEb: 35.73273751079803 36.26945977251843 33.367651902349614
WSFDE: 18.967547482188102
FDEv, FDEp, FDEb: 24.395048414119227 16.602132518526005 20.269549720996242

How should I solve it

from grip.

@Bigtuo if you did not change the code, and directly run the "evaluation.py", you are expected to get a very bad result.
Because they have already clearly describe that "./test_eval_data/prediction_gt.txt is just for testing the code which is not the real ground truth.", which means using testing on a fake ground truth. Please submit your result to their website to get true error. Thanks.

from grip.

from grip.