Settings for evaluation about genre HOT 5 CLOSED

As for any other model in pytorch you can do .cuda()

from genre.

Dear Nicola,

Thank you for your response and interesting work. We are struggling to reproduce your results. Can you please share your evaluation code and also all the changes that affect the results, but have been included in this repository (e.g., the changes you mentioned in your response)?

I guess there should not be IP/copyright issues to share your evaluation code and GERBIL middleware, right?

from genre.

Hi!

If you can run the example, there is no problem with your GENRE setting AFAIK. The problem is in the setting:

1. as you can see from the examples we have a setting where we use a mention_trie and a mention_to_candidates_dict. I used them as not empty for the experiments in the paper. In particular, I used the entity universe from https://github.com/dalab/end2end_neural_el and the same from the mapping from mention to candidates. I also did some filerting (e.g., since the task is named entity linking and not entity linking in general, I removed all mentions that do not start with a capital letter).
2. I did not use the function get_entity_spans for evaluating with GERBIL. That function was added in the repo afterwards to demonstrate how to get predictions using GENRE. For instance, that function does not handle splitting the input into chunks when is too long which is needed for evaluating properly (since BART can handle at most 1024 tokens). I also had to modify some code (from https://github.com/dalab/end2end_neural_el and the GERBIL repository for enabling having the middleware and unfortunately we cannot release that code).

I hope this is helfpul.

from genre.

Thank you very much for your detailed answer. It is very clear.

Could I also ask about the question 2 above: Is there any setting to allow the GPU to be used for inference? (By default, GPU is not used for inference, right?)

from genre.

Thanks!

from genre.