Comments (6)
This is something we are looking to implement in the near future, I was thinking that the multiple CPU requests should be made through the extended resource request for dataplane containers. This would ensure the scheduler also tracks the availability of dataplane cores as well.
Is that something you considered or is there a particular reason you chose to go fro a new option in the isolate command?
from cpu-manager-for-kubernetes.
Basically adding the option to the isolate command was just easy and quick to do. We needed something for testing.
I was also thinking about the ER request but so far I haven't found a way to get that information to the isolate command. That needs some more investigation.
from cpu-manager-for-kubernetes.
Ok, so, I haven't had time to write a proper doc so far, but maybe this suffices for the time being to discuss what I had in mind for CMK (making it a DP, basically).
If we generally agree this would be a good thing, we could in this direction instead:
"- Cmk init would be done by the Device Plugin. Reports the pools as consumable resources, with the number of cores available
We can decide to still maintain pool config on the host filesystem, but IMHO it would be beter to store this in CRDs, in the API server. Could be a further enhancement
- DP would also do the tainting, and labelling of the node
- An Admission plugin would be also part of the solution. Whenever a Pod is submitted to the API server asking for a CMK managed resource, the Admission plugin would mutate its specification
o First, it would add a sidecar container tothe Pod. Side car could contain the cmk binary itself. This also could be a further enhancement, we could start with simply mutating the mounting of the binary into the Pod spec
o Depending on the chosen solution where to store CMK config, we would either read the CRDs, or we would also mount the config filesystem into the Pod as right now
o Second, Admission controller would also overwrite the containers entrypoint with „cmk isolate <ORIGINAL_ENTRYPOINT>”
o Thirdly, we would also mutate in the „limit+request=1mCPU” in the Pod spec. This would kind of limit what Kubernetes thinks about the Pods CPU usage
Alternatively, or in addition to this, we think kubelet CPU manager should really respect isolcpus anyway, and should not include these cores into its own CPU set(s)
o Lastly: parameters to the cmk isolate should be put together by the DP, and communicated to the container.
Mutating config file mounting, environment variable source, or exposing these in the form of CRDs I think would all work. Then, the cmk isolate command wrapper would be added to the overwritten entrypoint with the right, Pod specific parameters - When a Pod is scheduled, DP would simply decrease the number of available cores in the pool of the node according to the request.
When the Pod exits, it increases it. This can be tricky right now as tehre is no „Release()” call in the DPAPI, but I think multiple people are trying to bring-it in - DP would also run periodically run reconcile, and update the Device Manager
"
Thoughts? This is still very draft and high-level, and was written with very limited CMK and DPAPI knowledge :)
from cpu-manager-for-kubernetes.
CMK should support allocating multiple cpus to a container. The support is needed for dataplane (exlusive) pool allocations.
Can you achieve this by simply configuring your pools to contain multiple CPUs? See doc.
from cpu-manager-for-kubernetes.
Can you achieve this by simply configuring your pools to contain multiple CPUs? See doc.
I think what we want is to assign more than one -but configurable amount of- CPUs from a pool, to the same container. So let's say we configure a pool with 8 CPUs, and then we want to assign 3 exclusive ones to Pod1, and then 5 to Pod2.
Ultimately we also might want to do something like this:
- assign 3 exclusive CPUs from Pool1 to Pod1
- assign 4 CPU worth of time share(!) from Pool2 to Pod1
from cpu-manager-for-kubernetes.
I opened two PRs that may be interesting for you.
#196 adds support for allocating multiple CPU cores, but instead of additional command line parameter it uses env variable.
#195 adds a mutating admission webhook server - initial idea was to read ER request and inject number of requested cores as env variable (to use it with #196), but then it grew up a bit and in current implementation volumes, volume mounts, host /proc fs variable etc. are injected as well, so it's partially aligned with @Levovar idea I believe.
from cpu-manager-for-kubernetes.
Related Issues (20)
- Only one node picked up by cluster-init which also fails with Forbidden Error HOT 1
- Cannot build CMK Docker image
- Issue about using hyper-thread in exclusive mode HOT 4
- Pin CPU for Pod on K8s
- INTERNAL ERROR: cannot create temporary directory! HOT 2
- can not run on the ARM v8 CPUs
- operator.md incorrectly states to set --insecure to false to revert back to regular TLS
- CMK Helm Chart Guidelines HOT 1
- CMK Configmap is hard to manage HOT 13
- CMK on Cluster Multi Node environment HOT 5
- CMK Isolation on different NUMA nodes HOT 6
- CMK crashes on Intel® Xeon® Processor E5-2690 HOT 1
- Error getting cmk-config-<node-name> Configmap HOT 2
- cmk-cluster-init-pod failed with CreateContainerConfigError HOT 3
- CMK reconfiguration function errors
- keyerror observed while running isolate command HOT 3
- Processes in shared pool not actually sharing the cores
- CMK resources are not showing after upgrading k8s
- Different pools for different containers
- CPU isolation from host daemons without isolcpus
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cpu-manager-for-kubernetes.