Comments (5)
@wujianming1996 尝试注释掉llama_server.py的第13行
from linly.
可以尝试一下:https://github.com/fengyh3/llama_inference
上述脚本是针对tencentpretrain的llama做inference的~
至于你提到的问题,我之前也试过,大概率是运行内存不够的问题。我运行内存是14G,然后加了24G的swap内存,没问题。
from linly.
啊。。我那台GPU是32G的内存。。。GenerateLm运行完毕后直接吃掉95%,然load_model,我分配了20G的虚拟内存,全部吃光然后被kill。。。分配30G虚拟内存还没试。。。腾讯云T4 GPU
from linly.
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
请问在部署微服务的时候如果没有GPU可以用CPU吗?
from linly.
@fengyh3 谢谢,我试试。
from linly.
Related Issues (20)
- Chinese-LLaMA-33B在多少块gpu上训了多长时间?
- Are the tokenizer.model the same with the one in llama-7b?
- huggingface上openllama-13b的模型大小为26.4G,转换为huggingface那种模型格式之后模型大小为24.7G,这也就是大概是以fp16或者是bf16保存的模型
- ChatFlow-13B.bin只有136字节 HOT 1
- python3 llama_server.py结果乱码
- 多轮对话问问题之后直接报错
- 微信满员了,请重新上传新的微信图片 我可以免费做管理员 HOT 3
- Please clarify the License for Chinese-LLaMA-2 HOT 1
- 关于Chinese-LLaMA-2-13B (hf格式)
- 请问,deepspeed 微调时,CPU的内存需要多大? HOT 1
- Chinese-LLaMA-2-13B-hf样本模板prompt到底是什么样的?
- readme上的加群二维码过期了
- 问下大佬们有没有训练3B的打算?场景需要时延不能太高
- 有人有pile的数据集吗?22个来源,825G的那个版本
- 服务器最低配置要求是什么?
- 在线地址无法使用
- pretrain.py的示例似乎有点错误
- 请问70B的模型要如何使用,抱脸上的模型看着文件和其他模型不一样
- 请问有没有性别年龄检测模型?
- llama3增量预训练冻结哪些层训练哪些层效果比较好?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from linly.