Comments (3)
Tejaswgupta
commented on May 8, 2024
I would like to contribute
from deepspeed.
jeffra
commented on May 8, 2024
Hi @Tejaswgupta, thanks for reaching out and wanting to contribute. Is there something specific you have in mind you'd like to contribute? If it was this issue I do have a PR being reviewed currently that fixes this issue. We're in the process of trying to create some new issues that would be good to take care of as a new user. Keep an eye out for those :)
@dapurv5, Thanks for bringing this bug to our attention. We previously fixed this type of bug in a different code path but now have added unit tests for it and fixed it in your code path and one other. The PR should be merged soon.
from deepspeed.
Tejaswgupta
commented on May 8, 2024
Sure @jeffra :)
from deepspeed.
Related Issues (20)
- [BUG] Does deepspeed MoE support dynamic network (with unused parameters)? HOT 5
- [BUG] Pipeline Parallel: Only few trainable parameters, Only GPU0 has parameters whose requires_grad is "True". Cause ValueError: optimizer got an empty parameter list when deepspeed.initialize
- [BUG]Loss decreases cyclically with epoch as the cycle HOT 1
- [BUG] 'type:transformer' partitioning doesn't ensure non-zero parameters on each pipeline rank. HOT 5
- [BUG] Profiler records values that differ from transformers
- [BUG] Problems with MiCS training HOT 1
- [BUG]Pipeline Parallel: engine.train_batch(data_iter=training_dataloader) HOT 1
- [BUG] deepspeed_aio swap file name too long. HOT 3
- [BUG] Error: Sizes of tensors must match except in dimension 1. Expected size 64 but got size 512 for tensor number 2 in the list. HOT 4
- A possible solution to resolve the issue of deepspeed.initialize() hanging.
- nv-nightly CI test failure HOT 1
- {{ env.GITHUB_WORKFLOW }} CI test failure
- [BUG] Error: Preparing DeepSpeed ZeRO stage 2 optimizer HIP error on AMD MI200 GPUs
- Confusing memory usage estimations
- [BUG] Unexpected GPU memory consumption when using transformers PEFT in Zero3
- [REQUEST] ZeRO - introduce replicas to keep GBS from getting too large on hundres of gpus HOT 4
- How to offload/move model weights manually ?
- Changing offload to NVMe instead of CPU causes error: TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' HOT 2
- [BUG] TypeError: Can't instantiate abstract class MPS_Accelerator with abstract methods HOT 1
- [BUG] main_process_ip not working HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepspeed.