Giter VIP home page Giter VIP logo

duee's Introduction

Hi there 👋

Visitor Count

Github Stats

About me

NLP算法工程师一枚🤓 多实践、多交流、多思考

公众号: NLP煎饼摊
微信: ZHOU-JXX
知乎
谷歌学术

duee's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

duee's Issues

DuEE缺少data文件夹

问下数据处理的时候调用的from data.data_utils import schema_process, data_process
data这个文件夹不存在,能发下么

f1分数

有的预测结果event_list是空的,提交上去计算出来的f1值都是0,请问这两者之间有关系吗,还是说提交文件的格式有错误

运行duee predict_cls出现的问题

运行:

CUDA_VISIBLE_DEVICES=2 python predict_cls.py --dataset=DuEE-Fin --event_type=enum -- max_len=256 --per_gpu_eval_batch_size=32 --model_name_or_path=/home/user/pretrained-model/chinese-roberta-wwm-ext-large --fine_tunning_model_path=./output/DuEE-Fin/enum/best_model.pkl --test_json=./data/DuEE-Fin/sentence/test.json

报错:
predict_cls.py: error: the following arguments are required: --fine_tunning_model_path, --test_json

关于标签问题

请问您复现的时候对于重叠标签是怎么处理的呢?是舍弃吗?

标签数据下标不对。

您好!我在复现时碰到以下问题:
File "run_ner.py", line 179, in
main()
File "run_ner.py", line 161, in main
eval_p, eval_r, eval_f1, eval_loss = evaluate(args, eval_iter, model, metric)
File "run_ner.py", line 46, in evaluate
n_infer, n_label, n_correct = metric.compute(batch["all_seq_lens"], preds, batch['all_labels'])
File "/home//DuEE/metric/metric.py", line 74, in compute
] for sent_index in range(len(lengths))]
File "/home/
/DuEE/metric/metric.py", line 74, in
] for sent_index in range(len(lengths))]
File "/home/***/DuEE/metric/metric.py", line 73, in
for index in labels[sent_index][:lengths[sent_index]]
KeyError: -1

打印出来label
[[-1 26 26 ... -1 -1 -1]
[-1 26 26 ... -1 -1 -1]
[-1 26 26 ... 26 26 26]
...
[-1 26 26 ... -1 -1 -1]
[-1 26 26 ... -1 -1 -1]
[-1 26 26 ... -1 -1 -1]]

对比一下,除了与paddle的ChunkEvaluator类中,相关的下标不同以外
image
好像没有其他区别。
请问有没有解决方法?

数据处理部分

您好,请问 doc["event_list"]为一个dict:有'trigger'、'event_type'、'arguments' ,对应的值是用什么方法识别的呢

单机多卡如何设置

你好~
按照你的步骤,源代码很轻松就跑下来了,比paddle版本结构更清晰,非常感谢~
后来我在这个版本的基础上,加lstm_crf层以后,状况就比较多了~提示内存不足,所以我就考虑改为多卡运行。因为我的服务器有2个GPU,多卡部分代码如下:

model = bert_lstm_crf(args.model_name_or_path, args.id2label,num_classes=args.num_classes,
rnn_hidden_size=args.rnn_hidden_size,rnn_layers=args.rnn_layers)
    model.cuda()
    net = nn.DataParallel(model)
    model=net.module
    # model.to(args.device)

但是运行过程中,GPU的利用率情况是下面这样的:
QQ截图20211115104944

依旧是1个GPU在跑,而且显存已经快爆了,但是利用率却不高。另外一个GPU一直不动。在你的代码里,有一个参数是do_distri_train,跟它有关系吗?
求帮忙,谢谢~~

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.