Hyperthreading awareness? about cpu-manager-for-kubernetes HOT 8 CLOSED

All requests, as you call them, are HT-aware (if HT is enabled) - allocation of a single core in CMK means you're allocating two threads running on the same physical CPU core, so there's no possibility that two separate workloads requesting resources will land on the same physical core.

If you're asking about potential races in general, please see Known Issues.

from cpu-manager-for-kubernetes.

Thanks @przemeklal. Does that mean my application should always allocate cores in multiples of two from a pool to ensure no noisy neighbors on other allocations?

Separate question, does cmk configure all the cpusets properly if all applications launch through it? A problem I have now is calico and other system pods float to isolcpus cores. If I wait for 1.15 would the new topology manager do this without cmk?

from cpu-manager-for-kubernetes.

Yes, in fact your app will always allocate two threads per single requested core on HT-enabled system, however you don't have to use all of them - for example you could run your workloads on a single thread per physical CPU and leave the other one unused without anything running there, avoiding noisy neighbor effect.

Regarding the second question - CMK doesn't have any mechanisms for checking whether anything runs on isolated cpus - it just assumes that it is the one and only consumer of isolated cores. Anyway, nothing but kworkers should run on the isolated cpus.
On the topology manager question - I'm afraid I can't answer to that, perhaps @lmdaly could provide more info.

from cpu-manager-for-kubernetes.

@przemeklal to achieve an isolcpus effect, should we just make two pools: one with the isolated CPUs, and another without, and ensure that non-RT jobs run on the non-isolated pool?

from cpu-manager-for-kubernetes.

@przemeklal it actually sounds like a pod is not able to make a request for multiple isolated cores on different sockets. For example, if I have a pod that wants to use 3 isolated cores from socket-id 0, and 3 from 1, that wouldn't be possible today.

from cpu-manager-for-kubernetes.

Yes, this way we ensure that performance sensitive workload doesn't have to send any data across NUMA nodes through the QPI, which could slow it down.

If you're really determined you could request 6 cores in the pod spec, then inside that pod overwrite CMK_NUM_CORES variable (assign 3), and run cmk isolate --socket-id=0 and cmk isolate --socket-id=1 in parallel. You'd need have 2 processes that "do something", but if you want to have 3 cores from each socket for a single process, I can imagine you could just record env output to get cpus assigned by CMK and do sleep infinity inside cmk isolate context, perhaps use --no-affinity flag as well. But, you know, this is just a very dirty hack and I guess not something we recommend for use in production.

from cpu-manager-for-kubernetes.

@przemeklal understood. Our application is very NUMA aware, and would never send data across sockets, even if given cores to do so. Because of NIC affinity to our pod, we must get cores from specific sockets that map to the NIC device, or we'd have the issue of going over UPI. If we make a pull request to ask for cores from each socket, is that something others would find useful? I'd imagine something like --core_req=0,5 --core_req=1,10 would be a request of 5 cores from socket 0, and 10 from socket 1.

from cpu-manager-for-kubernetes.

Moved to new issue

from cpu-manager-for-kubernetes.