Allows accounting based on resource.requests instead real usage about kube-opex-analytics HOT 9 CLOSED

Hi @caruccio ,
Thank you to report this.
Make sense, we'll investigue how to integrate this feature.

from kube-opex-analytics.

@rchakode I've tried doing it but not sure if in the right direction: caruccio@3357ad3

from kube-opex-analytics.

Dear @caruccio
Pleased to hear that you start contributing this feature.
Considering the actual design, I think there is not need to of extra databases:

Indeed, at each time we have:

• actual usage for cpu and memory
• usage efficiency (i.e. rq_eff = usage / request) => request = usage / rq_eff

So from my point of view, implementing accounting for requests is a matter of computation without additional databases.
=> A suggestion is to update the dump_histogram_analytics function to implement that.
=> It may be needed to add a specific "dump_histogram_analytics" if the update makes the code complex.

Let me know if you have further comments.

R.

from kube-opex-analytics.

Hey @rchakode. Thanks for your reply.
That's an elegant solution. However, since classes K8sUsage and Rrd are only "connected" by the databases, I can't see how to correlate usage and rf other than using its timestamps.

from kube-opex-analytics.

@caruccio
yes, timestamps is the way to go if you follow this path:.

• on the one hand, you will have a map of usage accounting as [resource_type][namespace][timestamp] => usage
• on the other hand, you will have a map of request efficiency as [resource_type][namespace][timestamp] => rf

Then you can compute a map of request accounting as [resource_type][namespace][timestamp] => usage/rf

Look like it's functional programming, but if you are not an expert on that, like me, just iterate with loops :)

Edit 1: remark that the dump_histogram_analytics method is static (expecting a RRD file) => not binding between the class (K8sUsage ) nor the database is required.

Edit 2: One of our team mates can help implement this feature with a high priority, let me know if your company can sponsor that.

Does this make sense for you?

from kube-opex-analytics.

Hey @rchakode.

The issue is that usage accounting stores ratios only, not absolute resource values. Since cluster capacity is lost on dumping, it can't be used on a cluster with autoscaling on (which is a must have for my case).

No need for high prio implementation for now. I'm doing an investigation only.

from kube-opex-analytics.

Ok.
But the point I want to understand : how do you want to charge requests without using ratio?

Do you mean, for example, you want to charge 1GB of memory at a given rate, instead of charging a portion of memory based on requests ?

from kube-opex-analytics.

Hey @rchakode. Sorry for the delayed response.

After some consideration, the current method of charging for a ratio of the total requested memory is enough for my needs.
Just submitted PR #66. Hope it's ok to merge.

Any question please feel free to reach.

from kube-opex-analytics.

Closed by #66

from kube-opex-analytics.