Comments (8)
All requests, as you call them, are HT-aware (if HT is enabled) - allocation of a single core in CMK means you're allocating two threads running on the same physical CPU core, so there's no possibility that two separate workloads requesting resources will land on the same physical core.
If you're asking about potential races in general, please see Known Issues.
from cpu-manager-for-kubernetes.
Thanks @przemeklal. Does that mean my application should always allocate cores in multiples of two from a pool to ensure no noisy neighbors on other allocations?
Separate question, does cmk configure all the cpusets properly if all applications launch through it? A problem I have now is calico and other system pods float to isolcpus cores. If I wait for 1.15 would the new topology manager do this without cmk?
from cpu-manager-for-kubernetes.
Yes, in fact your app will always allocate two threads per single requested core on HT-enabled system, however you don't have to use all of them - for example you could run your workloads on a single thread per physical CPU and leave the other one unused without anything running there, avoiding noisy neighbor effect.
Regarding the second question - CMK doesn't have any mechanisms for checking whether anything runs on isolated cpus - it just assumes that it is the one and only consumer of isolated cores. Anyway, nothing but kworkers should run on the isolated cpus.
On the topology manager question - I'm afraid I can't answer to that, perhaps @lmdaly could provide more info.
from cpu-manager-for-kubernetes.
@przemeklal to achieve an isolcpus effect, should we just make two pools: one with the isolated CPUs, and another without, and ensure that non-RT jobs run on the non-isolated pool?
from cpu-manager-for-kubernetes.
@przemeklal it actually sounds like a pod is not able to make a request for multiple isolated cores on different sockets. For example, if I have a pod that wants to use 3 isolated cores from socket-id 0, and 3 from 1, that wouldn't be possible today.
from cpu-manager-for-kubernetes.
Yes, this way we ensure that performance sensitive workload doesn't have to send any data across NUMA nodes through the QPI, which could slow it down.
If you're really determined you could request 6 cores in the pod spec, then inside that pod overwrite CMK_NUM_CORES
variable (assign 3), and run cmk isolate --socket-id=0
and cmk isolate --socket-id=1
in parallel. You'd need have 2 processes that "do something", but if you want to have 3 cores from each socket for a single process, I can imagine you could just record env
output to get cpus assigned by CMK and do sleep infinity
inside cmk isolate
context, perhaps use --no-affinity
flag as well. But, you know, this is just a very dirty hack and I guess not something we recommend for use in production.
from cpu-manager-for-kubernetes.
@przemeklal understood. Our application is very NUMA aware, and would never send data across sockets, even if given cores to do so. Because of NIC affinity to our pod, we must get cores from specific sockets that map to the NIC device, or we'd have the issue of going over UPI. If we make a pull request to ask for cores from each socket, is that something others would find useful? I'd imagine something like --core_req=0,5 --core_req=1,10 would be a request of 5 cores from socket 0, and 10 from socket 1.
from cpu-manager-for-kubernetes.
Moved to new issue
from cpu-manager-for-kubernetes.
Related Issues (20)
- Only one node picked up by cluster-init which also fails with Forbidden Error HOT 1
- Cannot build CMK Docker image
- Issue about using hyper-thread in exclusive mode HOT 4
- Pin CPU for Pod on K8s
- INTERNAL ERROR: cannot create temporary directory! HOT 2
- can not run on the ARM v8 CPUs
- operator.md incorrectly states to set --insecure to false to revert back to regular TLS
- CMK Helm Chart Guidelines HOT 1
- CMK Configmap is hard to manage HOT 13
- CMK on Cluster Multi Node environment HOT 5
- CMK Isolation on different NUMA nodes HOT 6
- CMK crashes on Intel® Xeon® Processor E5-2690 HOT 1
- Error getting cmk-config-<node-name> Configmap HOT 2
- cmk-cluster-init-pod failed with CreateContainerConfigError HOT 3
- CMK reconfiguration function errors
- keyerror observed while running isolate command HOT 3
- Processes in shared pool not actually sharing the cores
- CMK resources are not showing after upgrading k8s
- Different pools for different containers
- CPU isolation from host daemons without isolcpus
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cpu-manager-for-kubernetes.