Add support for allocating multiple cpus to a container about cpu-manager-for-kubernetes HOT 6 OPEN

This is something we are looking to implement in the near future, I was thinking that the multiple CPU requests should be made through the extended resource request for dataplane containers. This would ensure the scheduler also tracks the availability of dataplane cores as well.

Is that something you considered or is there a particular reason you chose to go fro a new option in the isolate command?

from cpu-manager-for-kubernetes.

Basically adding the option to the isolate command was just easy and quick to do. We needed something for testing.

I was also thinking about the ER request but so far I haven't found a way to get that information to the isolate command. That needs some more investigation.

from cpu-manager-for-kubernetes.

Ok, so, I haven't had time to write a proper doc so far, but maybe this suffices for the time being to discuss what I had in mind for CMK (making it a DP, basically).

If we generally agree this would be a good thing, we could in this direction instead:
"- Cmk init would be done by the Device Plugin. Reports the pools as consumable resources, with the number of cores available
We can decide to still maintain pool config on the host filesystem, but IMHO it would be beter to store this in CRDs, in the API server. Could be a further enhancement

• DP would also do the tainting, and labelling of the node
• An Admission plugin would be also part of the solution. Whenever a Pod is submitted to the API server asking for a CMK managed resource, the Admission plugin would mutate its specification
  o First, it would add a sidecar container tothe Pod. Side car could contain the cmk binary itself. This also could be a further enhancement, we could start with simply mutating the mounting of the binary into the Pod spec
  o Depending on the chosen solution where to store CMK config, we would either read the CRDs, or we would also mount the config filesystem into the Pod as right now
  o Second, Admission controller would also overwrite the containers entrypoint with „cmk isolate <ORIGINAL_ENTRYPOINT>”
  o Thirdly, we would also mutate in the „limit+request=1mCPU” in the Pod spec. This would kind of limit what Kubernetes thinks about the Pods CPU usage
  Alternatively, or in addition to this, we think kubelet CPU manager should really respect isolcpus anyway, and should not include these cores into its own CPU set(s)
  o Lastly: parameters to the cmk isolate should be put together by the DP, and communicated to the container.
  Mutating config file mounting, environment variable source, or exposing these in the form of CRDs I think would all work. Then, the cmk isolate command wrapper would be added to the overwritten entrypoint with the right, Pod specific parameters
• When a Pod is scheduled, DP would simply decrease the number of available cores in the pool of the node according to the request.
  When the Pod exits, it increases it. This can be tricky right now as tehre is no „Release()” call in the DPAPI, but I think multiple people are trying to bring-it in
• DP would also run periodically run reconcile, and update the Device Manager
  "

Thoughts? This is still very draft and high-level, and was written with very limited CMK and DPAPI knowledge :)

from cpu-manager-for-kubernetes.

CMK should support allocating multiple cpus to a container. The support is needed for dataplane (exlusive) pool allocations.

Can you achieve this by simply configuring your pools to contain multiple CPUs? See doc.

from cpu-manager-for-kubernetes.

Can you achieve this by simply configuring your pools to contain multiple CPUs? See doc.

I think what we want is to assign more than one -but configurable amount of- CPUs from a pool, to the same container. So let's say we configure a pool with 8 CPUs, and then we want to assign 3 exclusive ones to Pod1, and then 5 to Pod2.

Ultimately we also might want to do something like this:

• assign 3 exclusive CPUs from Pool1 to Pod1
• assign 4 CPU worth of time share(!) from Pool2 to Pod1

from cpu-manager-for-kubernetes.

I opened two PRs that may be interesting for you.

#196 adds support for allocating multiple CPU cores, but instead of additional command line parameter it uses env variable.
#195 adds a mutating admission webhook server - initial idea was to read ER request and inject number of requested cores as env variable (to use it with #196), but then it grew up a bit and in current implementation volumes, volume mounts, host /proc fs variable etc. are injected as well, so it's partially aligned with @Levovar idea I believe.

from cpu-manager-for-kubernetes.