Comments (5)
Please check your cuda devices .
from instantmesh.
I am using a single A800(80G), but I can only train it with batch_size=1, if I set batch_size=2, there also would be a cuda out of memory error.
from instantmesh.
Yes same thing, when I set batch_size=1 it works, but batch_size=2 it does not. However I am only missing a few GB (~2GB), so I was wondering if there is a way to optimize this? And also what happens if I want to distribute the training across multiple gpus, if I set batch_size=1, is it going to be 1 batch per GPU? Or the 1 batch will be distributed across the GPUs?
Because if it is a batch of size 1, then wouldn't we have issue with converging?
from instantmesh.
@Kev1MSL Hello, I encountered several problems in the training process, the structure of my dataset is as the picture says, but my training profile will not be written, I would like to ask for your help, thank you very much for your reply
from instantmesh.
@Kev1MSL hello,i am trying to run the training process,but i don't know how to construct the dataset ,can i have a look at the structure of dataset?thank you very much for your reply
from instantmesh.
Related Issues (20)
- Training time? HOT 1
- RuntimeError: handle_0 INTERNAL ASSERT FAILED at "../c10/cuda/driver_api.cpp":15, please report a bug to PyTorch.` HOT 2
- not all arguments converted during string formatting log HOT 2
- How to Improve Texture Quality and is there a way to tweak the model generation part by providing the normal maps?
- Torch cache not releasing
- Command '['ninja', '-v']' returned non-zero exit status 1 HOT 1
- 多卡运行 HOT 2
- Removal of logos/brand names HOT 3
- size and texture problem
- Camera Augmentation in the released training code
- 编译中断问题
- 怎么配置数据集文件
- What is I wish to generate the mesh with multiple images ? - I have front view, left view, right view and back view of a chair - how can I create a mesh utilising all these images ? HOT 13
- Cuda version error when duplicating space on Hugging Face
- could you provide the rendering scripts to evalutate ? HOT 10
- 8GB of models downloading every time I launch app.py
- can providing good quality of image will not impact quality HOT 2
- INTERNAL ASSERT FAILED after generating multiviews
- How to get the quantitative results as reported in the paper HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from instantmesh.