thu-keg / maven-dataset Goto Github PK
View Code? Open in Web Editor NEWSource code and dataset for EMNLP 2020 paper "MAVEN: A Massive General Domain Event Detection Dataset".
License: MIT License
Source code and dataset for EMNLP 2020 paper "MAVEN: A Massive General Domain Event Detection Dataset".
License: MIT License
Hi, I would like to submit to the leaderboard. I ran the BERT+CRF model and produced the result.jsonl
. But when I tried to submit to leaderboard at codalab I got the following issue
invalid data type ()
I am wondering what should I do in this case? Thank you!
关于DMBERT,是在已知trigger的情况下进行事件分类的吗,如何才能知道trigger是哪个呢
hello, 你好 negative_triggers 在数据集中的意义是什么啊
Hi, thanks for your work on this dataset.
I notice that you compare the performances of BiLSTM and BERT on both ACE2005 and MAVEN, and it seems that BiLSTM outperforms BERT on ACE2005. However, some papers report different results. For example, in https://www.aclweb.org/anthology/P19-1522/, they report a 80+ F1 score with BERT. And in https://www.aclweb.org/anthology/2020.emnlp-main.435/, results on BERT+MLP is better than DMBERT (76.2 vs 74.9). What do you think of these results?
作者您好,十分感谢希望您能回复我
语句是这样写的
for layer in range(self.num_layers):
rnn_fw, rnn_bw = self.rnns[layer]
output = dropout(outputs[-1], keep_prob=keep_prob, is_train=is_train)
with tf.variable_scope("fw_{}".format(layer)):
out_fw, state_fw = rnn_fw(output)
但是在最后语句, out_fw, state_fw = rnn_fw(output)报出错TypeError: call() missing 1 required positional argument: 'state'。我查找了好几天并未找到有效办法,请问您知道该如何解决么?
希望您能看到回复我
谢谢
Or the plan is to always submit to the leaderboard?
您好,请问数据集中分词使用的工具是什么呀,我使用斯坦福的分词工具,输出的token长度会出现不一致的问题
Just a quick question, as I cannot find the adversarial training in the DMBERT implementation.
It seems that it's just doing a sentence encoding and classification, without any generator/discriminator mechanism.
Is this implementation different from the one described in the original paper?
Thanks!
作者你好,我在论文5.1节第二段看到“Conventionally, the existing ED datasets only provide the standard annotation of positive instances (the annotated event triggers) and researchers will sample the negative instances (non-trigger words or phrases) by themselves, which may lead to potential unfair comparisons between different methods”
想请问作者,ACE数据集里是否也有这样的操作?如果有的话可否提供一两篇包含这种添加negative trigger做法的论文,因为自己目前在写毕业论文,想同时使用ACE和MAVEN数据集,但是两个数据集的测评方法不同,所以感到很棘手。十分感谢!
Hi, thanks for your great work, these baselines help me understand the event detection task!
When I run the code of BERT-CRF( I use 1 gpu, batch size = 16, Gradient Accumulation steps = 8), I get evaluation loss = nan, and p,r,f1-score = 0.
But when I turn the batch size to 2, it works fine.
So I don't know why batch size makes an impact on the result...😥could you help me figure out this confusion...
Do you train word2vec.npy model by yourself.
I don't see the details in your project introduction.
作者您好
我运行成功MOGANED模型得到了训练后dev的准确率值,召回率值,f1值,进行--eval得到result.jsonl。我应该怎样做才能得到最后测试集的结果呢??我看到有evaluate.py但是不能运行!
期待您的回复。
十分感谢,谢谢。
比如BERT+CRFbaseline在测评的时候没有使用trigger的标注信息。而DMBERTbaseline在测评的时候是按照trigger词将句子分成了两段。虽然test集中给定了trigger的offset,但是这两个模型显然放在一起比较不是很公平。官方比较推荐的测评方法是什么样的?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.