Information on models fine-tuning used in OmniEvent.infer about omnievent HOT 3 OPEN

Hello, that's a good question. The models we release are trained on multiple EE datasets. When training on different datasets, we add a prefix to represent the schema of the data. For example, we use "" to represent the schema of the Maven dataset. However, due to limitations in data volume and model capacity, the models we release sometimes struggle to follow human instructions (i.e., the schema prefix).
We are currently researching how to align the model better for IE tasks to make it more adept at following human instructions.

from omnievent.

Thank you for your answer.

I'm still unclear though on these prefixes. It seems like you did not add them as special tokens in the tokenizer. Is it that you considered that treating them like any other word was not a problem or am I missing something?

For instance in the following google colab notebook, the author does add their prefix "<idf.lang>" as a special token to the tokenizer: https://colab.research.google.com/github/KrishnanJothi/MT5_Language_identification_NLP/blob/main/MT5_fine-tuning.ipynb

from omnievent.

Thanks for the question. We trained two versions of the model: with prefixes added as special tokens or without. There is no significant difference between the results of these two. Previous work has also revealed the similar phenomenon (https://aclanthology.org/2022.aacl-short.21.pdf).

from omnievent.