The performance may vary depending on the environment, seed, and others.
I recommend using the pre-trained model for precise reproduction.
Thanks!

from wtal-uncertainty-modeling.

The performance may vary depending on the environment, seed, and others.
I recommend using the pre-trained model for precise reproduction.
Thanks!

Thank for your reply.I used the pre-trained model that you have updated to test,but the result is :

Step: 0
Test_acc: 0.8905
average_mAP: 0.4038
[email protected]: 0.6551
[email protected]: 0.5836
[email protected]: 0.5052
[email protected]: 0.4158
[email protected]: 0.3245
[email protected]: 0.2266
[email protected]: 0.1161

The result is worse than your paper.I think the envirment is not detemined with the test result.And the enviroment that I test your pre-trained model is in Google Colab.Is the pre-trained model you offerd newest?

from wtal-uncertainty-modeling.

The environment, in fact, affects the performance.
For instance, some numpy functions behave differently according to the numpy version.
Please ensure that your environment is identical to the requirements.
For the model file, I updated it last day.

from wtal-uncertainty-modeling.

The environment, in fact, affects the performance.
For instance, some numpy functions behave differently according to the numpy version.
Please ensure that your environment is identical to the requirements.
For the model file, I updated it last day.

Thank you for your quick reply.I will make the environment same to your requirements and test the model again.

from wtal-uncertainty-modeling.

The environment, in fact, affects the performance.
For instance, some numpy functions behave differently according to the numpy version.
Please ensure that your environment is identical to the requirements.
For the model file, I updated it last day.

Hello.
Thank you for your reply. I make the environment match your requirements,but I still can't reproduce the result .And the rsult is still as before:
Step: 0
Test_acc: 0.8905
average_mAP: 0.4038
[email protected]: 0.6551
[email protected]: 0.5836
[email protected]: 0.5052
[email protected]: 0.4158
[email protected]: 0.3245
[email protected]: 0.2266
[email protected]: 0.1161

And my environment is:

Name                    Version                   Build  Channel |   |   |   |   |  

_libgcc_mutex             0.1                        main |   |   |   |   |  
absl-py                   0.11.0                   pypi_0    pypi |   |   |   |   |  
astor                     0.8.1                    pypi_0    pypi |   |   |   |   |  
ca-certificates           2020.12.8            h06a4308_0 |   |   |   |   |  
cached-property           1.5.2                    pypi_0    pypi |   |   |   |   |  
certifi                   2020.12.5        py36h06a4308_0 |   |   |   |   |  
docopt                    0.6.2                    pypi_0    pypi |   |   |   |   |  
future                    0.18.2                   pypi_0    pypi |   |   |   |   |  
gast                      0.2.2                    pypi_0    pypi |   |   |   |   |  
google-pasta              0.2.0                    pypi_0    pypi |   |   |   |   |  
grpcio                    1.34.0                   pypi_0    pypi |   |   |   |   |  
h5py                      3.1.0                    pypi_0    pypi |   |   |   |   |  
importlib-metadata        3.3.0                    pypi_0    pypi |   |   |   |   |  
joblib                    0.13.0                   pypi_0    pypi |   |   |   |   |  
keras-applications        1.0.8                    pypi_0    pypi |   |   |   |   |  
keras-preprocessing       1.1.2                    pypi_0    pypi |   |   |   |   |  
ld_impl_linux-64          2.33.1               h53a641e_7 |   |   |   |   |  
libedit                   3.1.20191231         h14c3975_1 |   |   |   |   |  
libffi                    3.3                  he6710b0_2 |   |   |   |   |  
libgcc-ng                 9.1.0                hdf63c60_0 |   |   |   |   |  
libstdcxx-ng              9.1.0                hdf63c60_0 |   |   |   |   |  
markdown                  3.3.3                    pypi_0    pypi |   |   |   |   |  
ncurses                   6.2                  he6710b0_1 |   |   |   |   |  
numpy                     1.19.0                   pypi_0    pypi |   |   |   |   |  
openssl                   1.1.1i               h27cfd23_0 |   |   |   |   |  
opt-einsum                3.3.0                    pypi_0    pypi |   |   |   |   |  
pandas                    0.23.4                   pypi_0    pypi |   |   |   |   |  
pillow                    8.0.1                    pypi_0    pypi |   |   |   |   |  
pip                       20.3.3           py36h06a4308_0 |   |   |   |   |  
pqi                       2.0.6                    pypi_0    pypi |   |   |   |   |  
protobuf                  3.14.0                   pypi_0    pypi |   |   |   |   |  
python                    3.6.12               hcff3b4d_2 |   |   |   |   |  
python-dateutil           2.8.1                    pypi_0    pypi |   |   |   |   |  
pytz                      2020.4                   pypi_0    pypi |   |   |   |   |  
readline                  8.0                  h7b6447c_0 |   |   |   |   |  
scikit-learn              0.20.0                   pypi_0    pypi |   |   |   |   |  
scipy                     1.1.0                    pypi_0    pypi |   |   |   |   |  
setuptools                51.0.0           py36h06a4308_2 |   |   |   |   |  
six                       1.15.0                   pypi_0    pypi |   |   |   |   |  
sqlite                    3.33.0               h62c20be_0 |   |   |   |   |  
tensorboard               1.15.0                   pypi_0    pypi |   |   |   |   |  
tensorboard-logger        0.1.0                    pypi_0    pypi |   |   |   |   |  
tensorflow                1.15.2                   pypi_0    pypi |   |   |   |   |  
tensorflow-estimator      1.15.1                   pypi_0    pypi |   |   |   |   |  
termcolor                 1.1.0                    pypi_0    pypi |   |   |   |   |  
tk                        8.6.10               hbc83047_0 |   |   |   |   |  
torch                     1.6.0                    pypi_0    pypi |   |   |   |   |  
torchvision               0.7.0                    pypi_0    pypi |   |   |   |   |  
tqdm                      4.31.1                   pypi_0    pypi |   |   |   |   |  
typing-extensions         3.7.4.3                  pypi_0    pypi |   |   |   |   |  
werkzeug                  1.0.1                    pypi_0    pypi |   |   |   |   |  
wheel                     0.36.2             pyhd3eb1b0_0 |   |   |   |   |  
wrapt                     1.12.1                   pypi_0    pypi |   |   |   |   |  
xz                        5.2.5                h7b6447c_0 |   |   |   |   |  

And my cuda versionis 10.2:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

I don't know the reason that casue the problem.Maybe you change something that you ignored,but you forgot. I hope you can help me solve this problem.Because it' s very important for me.I'm waiting for your reply.
Thank you.

from wtal-uncertainty-modeling.

The environment, in fact, affects the performance.
For instance, some numpy functions behave differently according to the numpy version.
Please ensure that your environment is identical to the requirements.
For the model file, I updated it last day.

Hello.Can you tell me why the average result is 1.5 lower than your paper?Because it's very important for me.

from wtal-uncertainty-modeling.

Sorry, but I have no idea about what causes the performance difference, as I cannot look into the exact status of your environment.
I tried reproducing this repo on another environment and got the same result as mine.
A possible source is that you may be using the old code.
Make sure to use the latest code as well as the model file.

from wtal-uncertainty-modeling.

Sorry, but I have no idea about what causes the performance difference, as I cannot look into the exact status of your environment.
I tried reproducing this repo on another environment and got the same result as mine.
A possible source is that you may be using the old code.
Make sure to use the latest code as well as the model file.

Thank you for your reply.I have updated the newest code and the newest model.But the results don't change.Maybe you use some tricks that you ignored。

from wtal-uncertainty-modeling.

I'm sorry.
I found that the link to the pre-trained model was wrong, which is corrected now.
Please update the model file and re-evaluate the model.
Thanks!

from wtal-uncertainty-modeling.

I'm sorry.
I found that the link to the pre-trained model was wrong, which is corrected now.
Please update the model file and re-evaluate the model.
Thanks!

Thank you for your reply.
I can reproduce the result that used the newest pre-trained model now.Could you please update the code again? Maybe you forgot some details.
How long can you get the best model?

from wtal-uncertainty-modeling.

I'm sorry.
I found that the link to the pre-trained model was wrong, which is corrected now.
Please update the model file and re-evaluate the model.
Thanks!

Thank you for your reply.
I can reproduce the result that used the newest pre-trained model now.Could you please update the code again? Maybe you forgot some details.
How long can you get the best model?

Hi @xumh-9 , have you reproduced the result in paper? I tried many times, but I cannot get the result as good as pretrained model, even tried different random seeds.

from wtal-uncertainty-modeling.