Comments (3)
hiyouga
commented on August 16, 2024
用 deepspeed zero3 或者 fsdp
from llama-factory.
ChenBinfighting1
commented on August 16, 2024
请问是执行命令FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3.yaml就可以实现模型并行训练吗？
from llama-factory.
qy1026
commented on August 16, 2024
我后续没用llama-factory了。第一个参数我不知道是啥意思，强制数据并行？后面的代码是deepspeed_zero3+lora微调，应该是默认数据并行的。llama_factory好像不支持模型并行。
from llama-factory.
Related Issues (20)
- 训练了一批次数据后，使用export命令导出，可以接着该export导出的数据继续训练新的数据么？如果不能该怎么继续训练？
- DPO训练时，reward/acc很快收敛到1，但效果并不是很好 HOT 7
- 如何画不同数据的损失？
- 分布式训练的支持？ HOT 1
- 使用基于Llama-3-Chinese-8B-Instruct-v3做预训练，输出不会停止 HOT 2
- LLama-Factory怎么指定退火阶段加入特定训练数据 HOT 2
- Error when converting PiSSA adapter to normal LoRA in deepspeed stage 3 mode HOT 4
- template使用问题 HOT 2
- llama重复输出，在常见问题中未能解决 HOT 3
- NPU 910b4卡上推理速度很慢
- Low Learning Rate for Mistral Models
- 请教一下：想构建试题+试题问答的数据集来微调Qwen2
- ppo训练没有限制保存checkpoints数量功能。save_steps、save_strategy、save_total_limit没有任何作用。依然每步保存一个checkpoint。 HOT 3
- 长对话的微调 HOT 1
- LLava 1.5 7B 多模态在npu上进行推理进行问答默认是 HOT 1
- OSError: [WinError 126] 找不到指定的模块。 Error loading "D:\mysoft\anaconda3\envs\llama_factory\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies HOT 1
- 大佬们，如何使用vllm进行predict预测blue和rouge分数
- 大佬问一个大模型分词问题
- 训练完成之后结果非常不理想
- 可以支持DashInfer格式的大模型训练吗？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama-factory.