Comments (3)
Oh, I just found a way to see what process are running on the GPUs. In fact all 4 are running on the first GPU with none running on the others 3:
[Processes
Process ID : 80379
Type : C
Name : /data/ficklin/software/KINC-3.1.1/kinc-cli
Used GPU Memory : 210 MiB
Process ID : 80380
Type : C
Name : /data/ficklin/software/KINC-3.1.1/kinc-cli
Used GPU Memory : 210 MiB
Process ID : 80381
Type : C
Name : /data/ficklin/software/KINC-3.1.1/kinc-cli
Used GPU Memory : 210 MiB
Process ID : 80382
Type : C
Name : /data/ficklin/software/KINC-3.1.1/kinc-cli
Used GPU Memory : 210 MiB](url)
from kinc.
@bentsherman explained by email that current ACE/KINC defaults to always using the first GPU. And future version of ACE/KINC will balance out the load among the visible GPUs. So that makes sense now.
But, it seems SLURM will give me the GPU assigned to my job with the SLURM_JOB_GPUS environment
variable. KINC apparently sees all of the GPUs regardless how many I told it I wanted to use. So, I think even though future ACE/KINC will better divide up the workers by the visible GPUs, I will still need to provide the indexes to the GPUs that SLURM tells me I am allowed to run on.
Unfortunately, the format of the SLURM_JOB_GPUS is not defined in any documentation that I've looked at, so I'm not sure if it's list separated by commas or what...
from kinc.
I think you can also use the CUDA_VISIBLE_DEVICES
environment variable to control which GPUs can be seen by a node. More information here:
https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/
from kinc.
Related Issues (20)
- Kinc 3D viewer requires that the first column of the amx be labled "Sample" HOT 4
- KINC 3D viewer mangles TTY HOT 1
- Idea to Speed Similarity GMM Step
- Hypergeometric test performs poorly for sub-categories. HOT 3
- Documentation should include uninstall command
- Would be nice if we had a way to see version of kinc
- Need consistency of what files are called HOT 2
- Clarification on --minexpr and --nan values HOT 1
- Consistency of --preout and --postout HOT 1
- Ask for visualization dependencies for KINC HOT 5
- Philosophical thought about the use of "Source" and "Target" labels HOT 4
- install issue
- Need a "version" command HOT 1
- Additional Space prior to every positive correlation value HOT 1
- Post processing with KINC.R changes formatting HOT 1
- KINC 3D viewer attempts to read in p_value as character string
- KINC 3D viewer can not read in non-csGCN KINC network
- Log file print line at start
- Palmetto install broken HOT 2
- How to extract KINC network with container?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kinc.