Comments (8)
josStorer
commented on August 16, 2024
把site-packages\torch目录删了运行，会提示安装依赖, 安装指定版本的pytorch，需要用1.13
另外以下是2060 6G的推荐配置
from rwkv-runner.
josStorer
commented on August 16, 2024
保证 py310\Lib\site-packages\torch-1.13.1+cu117.dist-info 存在，不能是2.x版本
from rwkv-runner.
alphaDeng
commented on August 16, 2024
在安装新的pytorch（torch==1.13.1+cu117）以后，模型可以运行，但在上述设置下会出现爆显存的情况.
将载入显存层数降低到11层之后，此时模型能够运行，但是出现以下的警告，对话回复出现乱码的情况。
警告内容：
INFO: 127.0.0.1:52740 - "POST /switch-model HTTP/1.1" 200 OK
INFO: 127.0.0.1:52804 - "OPTIONS /chat/completions HTTP/1.1" 200 OK
INFO: 127.0.0.1:52804 - "POST /chat/completions HTTP/1.1" 200 OK
D:\py-ai\RWVK-Runner\py310\Lib\site-packages\rwkv\model.py:672: UserWarning: operator () profile_node %194 : int = prim::profile_ivalue(%192)
does not have profile information (Triggered internally at ..\torch\csrc\jit\codegen\cuda\graph_fuser.cpp:109.)
x, state[i5+0], state[i5+1], state[i5+2], state[i5+3] = ATT(
具体输出内容乱码情况：
切换为cpu时运作正常，这个或许是cuda版本问题？
from rwkv-runner.
josStorer
commented on August 16, 2024
不应该，我群里有同样用2060的，就用上面的配置，运行良好，比较奇怪
from rwkv-runner.
alphaDeng
commented on August 16, 2024
话说群里人用的2060的cuda是11.4以上的版本的吗？我这个好像是11.1的，目前nvdia-smi那里显示的cuda版本限制是11.4，我不太清楚现在能不能直接升级11.7的cuda。。。。
from rwkv-runner.
josStorer
commented on August 16, 2024
cuda是不用专门装的，更新显卡驱动就好了
from rwkv-runner.
alphaDeng
commented on August 16, 2024
嗯，目前用cuda的toolkit装了个cu11.7的cuda，那个套件顺手把驱动更新了。安装完以后可以正常运行了，但那个警告还是在那里。
from rwkv-runner.
josStorer
commented on August 16, 2024
那个警告无所谓，能正常用就好了
from rwkv-runner.
Related Issues (20)
- CPU/CPU(rwkv.cpp) 模型切换问题 HOT 1
- 调用api接口感觉速度有点慢，是否和流式输出有关系，我看接口记录中流式输出参数是false，这个"stream": false, 参数在哪里可以设置呢？ HOT 7
- 大佬，有兴趣加入Mirostat采样器吗？ HOT 1
- 如何完全卸载呢？ HOT 1
- 希望重新生成之前的对话时不会自动滚动到最后一段聊天
- JosStorer君，下载的东西都默认下载到C盘了。文件夹我是放到D盘的 HOT 1
- 模型运行成功，但发的所有消息都显示network error，控制台显示报错。 HOT 2
- 微调功能在上月新发布的 ubuntu 24.04 下失败
- 是否能够进行多batch的推理？
- install from source requirement.txt error on pkgutil
- 服务启动成功，但是访问报500 HOT 3
- 显卡能使用计算卡吗？ HOT 2
- 最新的rwkv-6-world模型转换后运行出错 HOT 1
- 因为ImportError: cannot import name 'packaging' from 'pkg_resources'导致模型转换失败 HOT 2
- 请教GPU负载均衡的问题 HOT 2
- lora微调训练问题 HOT 1
- linux部署的时候换一个rwkv模型无法运行
- Linux服务器的webGPU并行问题 HOT 1
- ERROR: Could not find a version that satisfies the requirement torch==1.13.1 (from versions: none) ERROR: No matching distribution found for torch==1.13.1
- RWKV依赖错误
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rwkv-runner.