Comments (7)
ydli-ai
commented on May 18, 2024
我们在制作数据集时伪造关键词部分已经排除了生成出的真关键词。标签为0时序列中至少有一个关键词是伪造的
…________________________________
发件人: poke ***@***.***>
发送时间: Friday, April 9, 2021 11:28:41 AM
收件人: CLUEbenchmark/CLUE ***@***.***>
抄送: Subscribed ***@***.***>
主题: [CLUEbenchmark/CLUE] CSL 样本噪声问题 (#115)
关键词识别任务，
”csl_public.zip 取自中文论文摘要及其关键词，论文选自部分中文社会科学和自然科学核心期刊。使用tf-idf生成伪造关键词与论文真实关键词混合，构造摘要-关键词对，机器学习模型的任务目标是根据摘要判断关键词是否全部为真实关键词“
存在一个问题：tf-idf生成的可能是真关键词，在训练集和验证集中发现了一些噪声：
[image]<https://user-images.githubusercontent.com/37020799/114124225-8a1e9880-9926-11eb-93b3-b208f49299ee.png>
测试集可能也有，如何处理这种噪声？能否公开关键词混合的方法？
―
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#115>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE3SPV5YSQMQ4PJQ6WDNJIDTHZX6TANCNFSM42UENV6Q>.
from clue.
Jasperty
commented on May 18, 2024
我们在制作数据集时伪造关键词部分已经排除了生成出的真关键词。标签为0时序列中至少有一个关键词是伪造的
…
我的截图，标出的那一行，标签=0，但是这些关键词在下面的1中都能找得到。
from clue.
Jasperty
commented on May 18, 2024
我们在制作数据集时伪造关键词部分已经排除了生成出的真关键词。标签为0时序列中至少有一个关键词是伪造的
重新截个图吧，原始数据集，第3行，标签=0，键长出现2次，应该有一个是tfidf构建出来的，关键词没有去重。实际上这行关键词全是真关键词。
from clue.
ydli-ai
commented on May 18, 2024
好的，我明白您的意思了，这个版本的数据集看上去确实有问题，我们在后续会发布新的CSL数据集，感谢您的提醒
…________________________________
发件人: poke ***@***.***>
发送时间: Friday, April 9, 2021 11:40:11 AM
收件人: CLUEbenchmark/CLUE ***@***.***>
抄送: Li Yudong ***@***.***>; Comment ***@***.***>
主题: Re: [CLUEbenchmark/CLUE] CSL 样本噪声问题 (#115)
我们在制作数据集时伪造关键词部分已经排除了生成出的真关键词。标签为0时序列中至少有一个关键词是伪造的
重新截个图吧，原始数据集，第3行，标签=0，键长出现2次，应该有一个是tfidf构建出来的，关键词没有去重。实际上这行关键词全是真关键词。
[image]<https://user-images.githubusercontent.com/37020799/114125010-2b5a1e80-9928-11eb-9526-89c7bb8f2bbb.png>
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#115 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE3SPV2ZAXFMX6XVWZALRNLTHZZJXANCNFSM42UENV6Q>.
from clue.
t1101675
commented on May 18, 2024
请问新的数据集有发布吗？在哪里发布呢？
from clue.
t1101675
commented on May 18, 2024
现在发现数据集里面还有其它噪声，比如验证集中 id 35 和 38 的两个样本，只是关键词的顺序换了一下，但是标签不一样。
from clue.
ydli-ai
commented on May 18, 2024
您好，数据集中存在少量噪声，但不影响数据集的难度和区分性，请基于当前版本的CSL进行测评。数据集如有更新会在CLUE上发布
…________________________________
发件人: brightmart ***@***.***>
发送时间: Saturday, May 8, 2021 11:19:06 PM
收件人: CLUEbenchmark/CLUE ***@***.***>
抄送: Li Yudong ***@***.***>; Assign ***@***.***>
主题: Re: [CLUEbenchmark/CLUE] CSL 样本噪声问题 (#115)
Assigned #115<#115> to @P01son6415<https://github.com/P01son6415>.
―
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub<#115 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE3SPV5YGVXN5GA3BIGWOITTMVI6VANCNFSM42UENV6Q>.
from clue.
Related Issues (20)
- csl提交榜单的分数和验证集的分数差距过大，为什么？
- DRCD数据集提交了没有成绩 HOT 1
- csl验证集跑到了90，到榜单只有84分，是为什么？ HOT 1
- CLUE排行榜好像挂掉了？ HOT 2
- 提交结果是否可以使用额外数据
- CLUE排行榜又挂掉了？
- 建议移除/baselines/models_pytorch/mrc_pytorch中对tensorflow的依赖
- 主站的数据集搜索挂掉了 HOT 1
- C3数据集怎么进行测试
- 提交失败 HOT 1
- CLUE benckmark 网站经常挂 HOT 1
- 提交失败
- pad_token_id不一致 HOT 2
- TensorArray Not Used on line 865 of tokenization_utils.py
- CLUE 榜单提交又挂了？？ HOT 2
- 提交失败 HOT 1
- Download Failure HOT 2
- 请求加入bard的测试 HOT 1
- 只对其中的某个子任务的数据集进行预测，如何得到其分数？
- 关键词识别 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clue.