MAP and MRR equals 0 about abcnn HOT 8 CLOSED

Hi,

How about retrying running the code after cleaning /models and /experiments folders?
There are some files made from my own experiments.

Actually, after training is done, models are saved automatically in /models and
after testing is done results are saved in /experiments.

I'll also try running the code in a different environment where I haven't developed the code.
(Please tell me which environment(Linux, windows, else..) you are using.)
Thank you.

from abcnn.

Actually,before running your code, I have cleaned models and experiments folders. I use ubuntu system, python 2.7, tensorflow 1.0.1. I just change the code "with open("./WikiQA_QA/Wiki-"+mode+".txt","r",encoding ="urged-8") as f" to "with open("./WikiQA_QA/Wiki-"+mode+".txt","r") " because python 2.7 doesn't support the former. Do you think the version of python will affect result? Thank you very much!

from abcnn.

Hi, Kim. I have another question. Did you consider the question about data imbalance? Because the number of unrelated QA pairs is much larger than related QA pairs. Thank you very much!

from abcnn.

Dear @jackyuanjie1990,
I want to tell you that I've verified the fact that my code runs well in other environments(I tested on Ubuntu 14.04.1, python3, only with CPU support.)
I highly recommend you to run the code on Python3 because there may be latent errors when you run the code written in Python 3 on Python 2 environments.

And I didn't consider the thing you mentioned(data imbalance problem) because that's the WikiQA dataset itself.
I didn't modify the dataset because I want to reproduce results as the authors suggested.

Thank you.

from abcnn.

Thank you for your patient reply! I will try Python3.

from abcnn.

Hi Kim. I get the same result as you post after I change python 2.7 to 3.5. You are right, it really has latent errors when run this code under python 2.7. Anyway, thank you very much again.

from abcnn.

The reason is that MAP and MRR is initialized to int 0(python 2.7)，so change it to float(0),and you can get the correct result.

from abcnn.

@wangzil thanks for your correction.

from abcnn.