Comments (5)
学习率设置多少合适呀?我这loss一直不降
from bert_seq2seq.
学习率设置多少合适呀?我这loss一直不降
一般1e-5即可,如果loss不下降甚至升高,考虑是不是代码哪里出问题了。
from bert_seq2seq.
谢谢,可是代码我没改呀,只有加载数据部分稍作改动。
from bert_seq2seq.
谢谢,可是代码我没改呀,只有加载数据部分稍作改动。
什么任务,哪个模型,而且你得打印下 确保输入输出是没问题的,并且看下验证的过程,看看是否能正常预测,不排除有时候某些模型loss确实是不会变化太大,但是能预测的很好。
from bert_seq2seq.
输入输出都没问题,模型训练过程中每N个step输出一下test_data的结果,看着也还可以。我试了3个模型,gpt2、roberta、bart,都是loss越来越大。我再调调参数吧~谢谢啦
from bert_seq2seq.
Related Issues (20)
- roberta_auto_title_train.py HOT 4
- 您好,我想添加一些我自己的数据集做fine-tune,请问应该怎么做呢 HOT 1
- 麻烦作者大佬同意一下入群申请,非常感谢! HOT 2
- 请问bart做标题生成怎么跑起来? HOT 4
- unilm多少epoch收敛比较好? HOT 1
- 请问在做标题生成的时候支持point_net对应的copy机制吗? HOT 1
- 你好,我想问一下,关于gpt文章写作的数据集应该是什么样的格式,是一小段标题作为输入,文章作为输出么? HOT 5
- 关于t5_auto_title_train.py
- 您好 请问在simbert模型里面 seq2seq的loss比similarity的loss大的多是正常的吗? HOT 3
- 您好! 关于您的simbert_train.py的动态padding部分 得到的token_type_ids有的就会类似于[0,0,0,0,1,1,1,0,0,0] 既然它是用来区分句子的,那到底拿1去padding还是0呢? HOT 2
- 自動摘要 fine tune HOT 1
- 关于细粒度NER数据集
- 关于UniLM HOT 1
- 多卡训练失败,目前有4个gpu,但使用的时候一直显示用的cuda0,指定其他cuda也不生效
- 数据长度
- 我从新改变了vocab.txt的大小,在加载预训练模型的时候出现维度不匹配,请问应该怎么解决了?我发现代码里面有个keep_tokens,似乎可以解决,但我不知道keep_tokens改如何设置。
- 请问英文的标题生成要怎么修改?
- 数据集分享
- 关于分词示例roberta_participle_CRF_train.py
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert_seq2seq.