thu-keg / maven-dataset Goto Github PK

Hi, I would like to submit to the leaderboard. I ran the BERT+CRF model and produced the result.jsonl. But when I tried to submit to leaderboard at codalab I got the following issue

invalid data type ()

I am wondering what should I do in this case? Thank you!

关于DMBERT，是在已知trigger的情况下进行事件分类的吗，如何才能知道trigger是哪个呢

hello, 你好 negative_triggers 在数据集中的意义是什么啊

Hi, thanks for your work on this dataset.
I notice that you compare the performances of BiLSTM and BERT on both ACE2005 and MAVEN, and it seems that BiLSTM outperforms BERT on ACE2005. However, some papers report different results. For example, in https://www.aclweb.org/anthology/P19-1522/, they report a 80+ F1 score with BERT. And in https://www.aclweb.org/anthology/2020.emnlp-main.435/, results on BERT+MLP is better than DMBERT (76.2 vs 74.9). What do you think of these results?

作者您好，十分感谢希望您能回复我
语句是这样写的
for layer in range(self.num_layers):
rnn_fw, rnn_bw = self.rnns[layer]
output = dropout(outputs[-1], keep_prob=keep_prob, is_train=is_train)
with tf.variable_scope("fw_{}".format(layer)):
out_fw, state_fw = rnn_fw(output)
但是在最后语句， out_fw, state_fw = rnn_fw(output)报出错TypeError: call() missing 1 required positional argument: 'state'。我查找了好几天并未找到有效办法，请问您知道该如何解决么？
希望您能看到回复我
谢谢

Or the plan is to always submit to the leaderboard?

您好，请问数据集中分词使用的工具是什么呀，我使用斯坦福的分词工具，输出的token长度会出现不一致的问题

Just a quick question, as I cannot find the adversarial training in the DMBERT implementation.

It seems that it's just doing a sentence encoding and classification, without any generator/discriminator mechanism.

Is this implementation different from the one described in the original paper?

Thanks!

求助：我在执行BERT+CRF中run_MAVEN.sh的时候，遇到maven/dev.jsonl文件不存在的问题。下载的数据集中只有train/test/valid.jsonl三个，并没有所谓的dev.jsonl，前面的过程正常执行。

Hi, thanks for your great work, these baselines help me understand the event detection task!

When I run the code of BERT-CRF( I use 1 gpu, batch size = 16, Gradient Accumulation steps = 8), I get evaluation loss = nan, and p,r,f1-score = 0.

But when I turn the batch size to 2, it works fine.

So I don't know why batch size makes an impact on the result...😥could you help me figure out this confusion...

Do you train word2vec.npy model by yourself.
I don't see the details in your project introduction.

作者您好
我运行成功MOGANED模型得到了训练后dev的准确率值，召回率值，f1值，进行--eval得到result.jsonl。我应该怎样做才能得到最后测试集的结果呢？？我看到有evaluate.py但是不能运行！
期待您的回复。
十分感谢，谢谢。

比如BERT+CRFbaseline在测评的时候没有使用trigger的标注信息。而DMBERTbaseline在测评的时候是按照trigger词将句子分成了两段。虽然test集中给定了trigger的offset，但是这两个模型显然放在一起比较不是很公平。官方比较推荐的测评方法是什么样的？