Comments (6)
zhangymyeah
commented on August 20, 2024
所以这样是出现问题吗？当我在部署模型的时候，链接服务就停在这里，一直动不了，但是不知道为什么，（禁用了GPU）
from codeshell-vscode.
JulyFinal
commented on August 20, 2024
所以这样是出现问题吗？当我在部署模型的时候，链接服务就停在这里，一直动不了，但是不知道为什么，（禁用了GPU）
这样是成功启动了的。动不了的问题我观察可能是vscode的插件有问题吧，这个很慢才但会响应，目前发现只有chat能用，自动补全完全没效果。
from codeshell-vscode.
4869APTX
commented on August 20, 2024
同样存在编译后运行错误的问题，错误信息如下
llm_load_print_meta: format = GGUF V2 (latest)
llm_load_print_meta: arch = codeshell
llm_load_print_meta: vocab type = BPE
llm_load_print_meta: n_vocab = 70144
llm_load_print_meta: n_merges = 72075
llm_load_print_meta: n_ctx_train = 8192
llm_load_print_meta: n_embd = 4096
llm_load_print_meta: n_head = 32
llm_load_print_meta: n_head_kv = 8
llm_load_print_meta: n_layer = 42
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_gqa = 4
llm_load_print_meta: f_norm_eps = 1.0e-05
llm_load_print_meta: f_norm_rms_eps = 0.0e+00
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 16384
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type = 7B
llm_load_print_meta: model ftype = mostly Q4_0
llm_load_print_meta: model params = 7.98 B
llm_load_print_meta: model size = 4.25 GiB (4.58 BPW)
llm_load_print_meta: general.name = CodeShell
llm_load_print_meta: BOS token = 70000 '<|endoftext|>'
llm_load_print_meta: EOS token = 70000 '<|endoftext|>'
llm_load_print_meta: UNK token = 70000 '<|endoftext|>'
llm_load_print_meta: PAD token = 70000 '<|endoftext|>'
llm_load_print_meta: LF token = 28544 'ÄĬ'
llm_load_tensors: ggml ctx size = 0.17 MB
llm_load_tensors: using CUDA for GPU acceleration
llm_load_tensors: mem required = 154.30 MB
llm_load_tensors: offloading 42 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 45/45 layers to GPU
llm_load_tensors: VRAM used: 4201.34 MB
.GGML_ASSERT: ggml-cuda.cu:6115: false
Aborted (core dumped)
from codeshell-vscode.
zhangymyeah
commented on August 20, 2024
这样是成功启动了的。动不了的问题我观察可能是vscode的插件有问题吧，这个很慢才但会响应，目前发现只有chat能用，自动补全完全没效果。
现在我和你情况完全一样
from codeshell-vscode.
4869APTX
commented on August 20, 2024
这样是成功启动了的。动不了的问题我观察可能是vscode的插件有问题吧，这个很慢才但会响应，目前发现只有chat能用，自动补全完全没效果。
cpu 模式下自动补全非常慢，还是能够成功出现自动补全结果的
from codeshell-vscode.
vimBashMing
commented on August 20, 2024
这样是成功启动了的。动不了的问题我观察可能是vscode的插件有问题吧，这个很慢才但会响应，目前发现只有chat能用，自动补全完全没效果。
- 目前自动补全功能已完善，已更新了server和插件。请参考README，更新本地的llama_cpp_for_codeshell，重新make，并更新插件。
- CPU运行大模型确实很慢，建议使用GPU或者M1/M2芯片及以上的Mac。
from codeshell-vscode.
Related Issues (20)
- 是否考虑 用template 来向LLM 提问？ HOT 2
- 启动TGI报错
- GPU方式运行模型服务出错 HOT 3
- -ngl 0 不加触发自动补齐服务就挂了，加了得10s+才能出来 HOT 4
- 回答问题后输出无法自动停止 HOT 1
- Docker启动量化模型的GPU模式 HOT 2
- FetchError:request to http://127.0.0.1:8080/completionfailed HOT 2
- 可以把 codeshell-chat-q4_0.gguf 往魔搭社区传一份吗？ HOT 2
- Is is support deepseek coder yet ? HOT 2
- RAG+Langchain to make its more powerful?
- 检查性能问题，跟检查安全问题的prompts函数混淆了 HOT 1
- 魔搭4it模型中缺codeshell-chat-q4_0.gguf HOT 2
- 插件中访问模型url添加后缀completion，模型那边部署是completions HOT 2
- docker run --gpus 'all' 报错，多卡不支持吗 HOT 1
- 模型服务启动后网页能正常访问，但是vscode插件访问时服务端提示404异常？
- code-server安装codeshell-vscode插件对话框空白 HOT 1
- 有没有在windows上本地部署（用GPU、不用docker）的办法
- unable to load model
- codeshll不支持shared吗
- codeshell支持idea的插件有开源吗？ 怎么没有看到地址
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from codeshell-vscode.