cluster-autoscaler clusterapi provider performance degrades when there are a high number of node groups about autoscaler HOT 4 OPEN

/area provider/cluster-api

from autoscaler.

i've been hacking on a PR to add some timing metrics on the NodeGroups interface function. i believe we spend the most time in this function and have been trying to prove out how the number of node groups affects the time that this call takes.

elmiko@1a5d9cd

from autoscaler.

I don't think kas calls are the main bottle neck but rather the cloudprovider.NodeGroup function implementation. Currently it takes ~20 seconds with ~90 MachineSets. This #6796 avoids expensive loop and copy pointers resulting in ~5 seconds each NodeGroups call .

from autoscaler.

it seems we might have multiple areas for improvement, when i observed behavior with 50 to 75 node groups, i could see the performance becoming worse over time. it appeared that we might have inefficiency in the way we handle all the various cluster-api CRs.

from autoscaler.