Comments (3)
#8371 已提pr
from paddlenlp.
你好!我已经找到了这个问题。Meta-Llama-3-8B-Instruct 在生成时,eos_token是另一special_token 即<|eot_id|>
,128009
。但是,在tokenzier中,并没有正确加载这个special_token。
我手动加入128009,可以成功让模型自然停止生成。
下面是麦当劳的例子。
messages = [
{"role": "system", "content": "You are an expert at planning marketing events outdoors for small to medium size diners and restaurants. "},
{"role": "user", "content": "Help a local McDonald restaurant plan a promotion event for the anniversary of Big Mac."},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pd"
)
terminators = [
tokenizer.eos_token_id,
# tokenizer.convert_tokens_to_ids("<|eot_id|>")
128009,
]
outputs = model.generate(
**input_ids,
max_new_tokens=1024,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
out = tokenizer.batch_decode( outputs[0] )
This plan should help create a fun and engaging event that will drive sales, increase brand loyalty, and generate buzz around the anniversary of the Big Mac.<|reserved_special_token_5|>
HF model card例子:
messages = [
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
{"role": "user", "content": "Who are you?"},
]
Arrrr, me hearty! Me name be Captain Chat, the scurviest pirate chatbot to ever sail the Seven Seas o' the Interwebs! Me and me trusty crew o' code be here to swab the decks o' yer queries and answer yer questions with a pirate's flair! So hoist the colors, me hearty, and let's set sail fer a swashbucklin' good time!<|reserved_special_token_5|>
<|reserved_special_token_5|>
应该为<|eot_id|>
。
我不了解paddlenlp如何加载多个config文件,请你们想办法把这个改一下吧,拜托了🙏。
from paddlenlp.
好的,我们检查一下
from paddlenlp.
Related Issues (20)
- Taskflow默认的最大序列长度怎么看?FastDeploy UIE中最长序列长度怎么设置? HOT 12
- [Question]: 2.8版本使用LLM工作流报错缺少fused_ln HOT 2
- [Bug]: pipelines中语义检索系统,启动运行后,上传扫描式PDF文件 无法解析 HOT 1
- [Bug]: TaskFlow zero_shot_text_classification HOT 3
- [Bug]: get_rank_by_dim_and_process_id 函数未实现
- [Question]: paddle.distributed.launch 启动多进程训练结束后Loading best model from checkpoint 报错 HOT 7
- 如何对长文本进行抽取 HOT 3
- uie可以做嵌套抽取吗? HOT 3
- 文档公式有误 HOT 5
- [Question]: 请问文档智能任务有用自己数据集微调的教程吗? HOT 1
- [Bug]: ImportError: DLL load failed while importing libpaddle: 找不到指定的程序。
- [Question]: 分布式
- [Question]: Data annotation and pre processing for Relation Extraction
- [Bug]: paddle的nansum不支持empty的求和
- [Bug]: Taskflow("document_intelligence"): Illegal instruction (core dumped) HOT 7
- [Bug]: AutoModel加载本地路径模型报错 HOT 2
- UTC做多标签零样本训练,测试出现过拟合怎么办?
- [Question]: 语义检索Pipelines,召回速度 HOT 1
- [Bug]:UIE-X-base模型微调报错 HOT 2
- taskflow和fastdeploy放在一起会产生中断,是怎么回事呢? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paddlenlp.