argonne-lcf / compperfworkshop Goto Github PK

Hi Folks,

I am trying to gain understanding of how GPU (space and time) sharing works in nvidia. I have gone through the docs pages and also performed some experiments on it.

It would be of immense help to me if a GPU expert here could clarify if my understanding is correct.

Sorry if it is not appropriate channel for the query. Please let me know if you would recommend a more appropriate channel for discussion

As per my understanding:

• For every CPU process an equivalent GPU context is created.

No MIG / MPS case (Normal)

• Without MIG or MPS in place, we can have only 1 context utilizing the GPU.
• However, a context, could use multiple streams to execute kernels asynchronously on GPU.
• If there are more than 1 context spawned, it is time shared and never space shared.

MPS

• MPS provides with a mechanism to chunk a GPU upto a maximum of 48 ways using Software segregation.
• However, this is limited only to SMs and not its memory or cache.
• Here we can divide SMs into decimal %ges at a process / context level.

MIG

• MIG is the latest feature that allows for a complete compute and memory isolation where a GPU can be diveded upto a maximum of 7 partitions.
• It promotes guaranteed performance

A few references:
https://stackoverflow.com/a/31643688/9328077
https://stackoverflow.com/a/14896945/9328077
https://www.nvidia.com/en-us/technologies/multi-instance-gpu/
https://docs.nvidia.com/deploy/mps/index.html

I am also searching for profilers that could give me insights from GPU standpoint and not from a process stand point Could you please share your inputs on this?

Hi Murali / LCF Community,

Thank you very much for your contribution on https://www.youtube.com/watch?v=nYI0yt3Q82Y. I loved the video and wanted to try it out myself.

Is there a place from where I can clone the repo? The current 2022 repo doesn't seem to have MIG code (a quick glance didn't land me one)

Could you please help with the same?

Thanks!