Comments (2)
shashankvasisht
commented on August 11, 2024
I am facing the same issue. Did u solve this?
@Maclory Pls look into this. I have 24GB memory in GPU and i am using patches of 128 even in validation time, so the input image size does not seem to be an issue. The training gives Out Of memory exactly after 4900 iterations just as @ahmadmobeen said in the above comment. I feel this is somehow related to gradient accumulation.
from spsr.
Leeyed
commented on August 11, 2024
it seems that due to "Branch_init_iters=5000", after 5000 iters some parameters of generator requires grad, which cost more GPU memery. So, you need to reduce your batch size.
from spsr.
Related Issues (20)
- 模型的训练是如何安排的？ HOT 3
- RuntimeError: stack expects each tensor to be equal size, but got [3, 0, 0] at entry 0 and [3, 164, 0] at entry 1
- 关于梯度图 HOT 2
- About evaluation toolbox configuration. HOT 1
- 关于root中的path HOT 3
- I am running out of memory on gpu HOT 3
- Errors accur when using evaluation tool box HOT 1
- Cannot reproduce the reported performance using the pretrained checkpoint HOT 1
- 训练时预处理的lmdb部分报错 HOT 1
- 您好，我在运行您的代码的时候遇到一个问题，期待您的答复，谢谢。在运行时，可以正常训练，但是验证时，会爆显存RuntimeError:CUDA out of memory。请问是什么原因呢 HOT 4
- Test with multiple GPUs HOT 3
- blank image HOT 6
- waring HOT 2
- get a Error while run generate_dataset.py.
- question about test HOT 3
- Evaluation code modification suggestion
- 关于使用工具箱测试自己的模型 HOT 1
- create model.py is not inside models folder
- About HR_size and CUDA memory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spsr.