machine-tom / bertsum-chinese-lai Goto Github PK

模型进行测试的时候，函数test_rouge报错 “assert len(candidates) == len(references)”，请问是什么原因呢？

想请问下 ，LCSTS数据量这么大，您训练了多少步啊？

现在这个模型是如何做推断的，有推断的入口吗？对于没有摘要标签的数据如何预测摘要？

为什么在类DataIterator中传给方法batch_buffer的参数batch_size是batch_size * 50?

Traceback (most recent call last):
File "train_LAI.py", line 340, in
train(args, device_id)
File "train_LAI.py", line 272, in train
trainer.train(train_iter_fct, args.train_steps)
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/trainer.py", line 142, in train
for i, batch in enumerate(train_iter):
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/data_loader.py", line 131, in iter
for batch in self.cur_iter:
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/data_loader.py", line 229, in iter
for idx, minibatch in enumerate(self.batches):
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/data_loader.py", line 215, in create_batches
for buffer in self.batch_buffer(data, self.batch_size * 50):
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/data_loader.py", line 198, in batch_buffer
ex = self.preprocess(ex, self.is_test)
File "/data/nlp/zhaiyao/bertsum/bertsum-chinese-LAI/src/models/data_loader.py", line 186, in preprocess
tgt_txt = ex['tgt_txt']
KeyError: 'tgt_txt'

There is a issue in data_ builder_LAI.py at line 78. In the inner function named "_rouge_clean", the regex expression r'[^a-zA-Z0-9 ]' is for the English and have issue for chineses texts.

[Errno 2] No such file or directory: '../bert_data\bert.pt_data\LCSTS.train.0.bert.pt'

你好，可以分享一下LCSTS2.0原始数据集吗？我按官方的填写了申请，一直没有收到他们的回信，CSDN的链接也失效的，多谢。[email protected]

您好！
我在训练的时候遇到了一个错误
Traceback (most recent call last):
File "train_LAI.py", line 339, in
train(args, device_id)
File "train_LAI.py", line 271, in train
trainer.train(train_iter_fct, args.train_steps)
File "E:\wyf\Projects\bertsum-chinese-LAI-master\src\models\trainer.py", line 142, in train
for i, batch in enumerate(train_iter):
File "E:\wyf\Projects\bertsum-chinese-LAI-master\src\models\data_loader.py", line 133, in iter
for batch in self.cur_iter:
File "E:\wyf\Projects\bertsum-chinese-LAI-master\src\models\data_loader.py", line 237, in iter
batch = Batch(minibatch, self.device, self.is_test)
File "E:\wyf\Projects\bertsum-chinese-LAI-master\src\models\data_loader.py", line 27, in init
src = torch.tensor(self._pad(pre_src, 0))
File "E:\wyf\Projects\bertsum-chinese-LAI-master\src\models\data_loader.py", line 14, in _pad
width = max(len(d) for d in data)
ValueError: max() arg is an empty sequence
我试着运行了您给的例子中的数据，也出现了同样的错误，请问这是json到.pt时，数据没有正确的进行转换的原因，还是读取数据时发生的错误呢？
此外，data_builder_AI.py 中的labels 是句子的标签吗？因为我输出后发现，有些列表中只有一个1，但我的n_corps=2
感谢您的回复！

Step 4出现这个错误

请问您在进行train的时候有出现这个报错吗

您好,感謝所做的成果.
我根據您的程式使用LCSTS2.0資料集執行完畢,使用50%資料,訓練10000次,執行 Train_LAI.py -mode test ......
輸出結果在-result_path的路徑下得到2個檔案LCSTS_stepxxx.candidate , LCSTS_stepxxx.gold
LCSTS_stepxxx.candidate應該是模型訓練完生成的摘要?
但是摘要內容的句子數量都跟輸入短文一樣多, 看起來只是順序不一樣
請問這是因為訓練次數不夠嗎?