How do we track inter AZ, internet ingress data charges at pod level?

Error explanation

Dears I found an issue in prometheus server when processing the rules that if two different machine types got the same ip and hence kubernetes node name, then the rule would fail as it does not know how to join the metrics.

Error Message (in prometheus server)

err="found duplicate series for the match group {node=\"ip-172-30-42-190.eu-west-1.compute.internal\"} on the right hand-side of the operation:
[
{label_beta_kubernetes_io_instance_type=\"m5d.2xlarge\", label_eks_amazonaws_com_capacity_type=\"ON_DEMAND\", node=\"ip-172-30-42-190.eu-west-1.compute.internal\"}, 
{label_beta_kubernetes_io_instance_type=\"m5d.xlarge\", label_eks_amazonaws_com_capacity_type=\"ON_DEMAND\", node=\"ip-172-30-42-190.eu-west-1.compute.internal\"}];
many-to-many matching not allowed: matching labels must be unique on one side"

My solution

I've developed node-to-pod-labeler that scans the whole cluster and as soon as a pod is assigned to a node it will transfer the relevant labels from the node to the pod, then kube-state-metrics will expose these labels as part of the metrics, then the rules can be simplified and this error avoided.

Im sharing it in case you find it useful

Due to deprecation of label_beta_kubernetes_io_instance_type
https://kubernetes.io/docs/reference/labels-annotations-taints/#beta-kubernetes-io-instance-type-deprecated

if possible update to label_node_kubernetes_io_instance_type in the labels of the instance cost metrics.

Thanks!

I have clusters I want to test this on, but none of them are in eu-west-1. I forked the repo and added this feature (with some hackey and really ugly golang) that gets the AWS_REGION from an environment variable. Please feel free to critique and provide suggestions as to how it can be better. I didn't create a PR, I wanted to see how it was received first.

Branch: https://github.com/mrballcb/kubernetes-cost-report/tree/aws-region-env-var
Commit: main...mrballcb:kubernetes-cost-report:aws-region-env-var

I built a docker image of the above and put it at:
https://hub.docker.com/repository/docker/mrball/cost-report

I am currently testing it on one of my clusters.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• Update actions/setup-go action to v5
• Update azure/setup-helm action to v4
• Update dependency ubuntu to v22
• Update docker/build-push-action action to v5
• Update docker/login-action action to v3
• Update docker/metadata-action action to v5
• Update docker/setup-buildx-action action to v3
• Update github/codeql-action action to v3
• Update goreleaser/goreleaser-action action to v6
• Update release-drafter/release-drafter action to v6
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update module golang.org/x/text to v0.3.8 [SECURITY]
• Update module google.golang.org/protobuf to v1.33.0 [SECURITY]
• Update WillAbides/setup-go-faster action to v1.14.0
• Update aquasecurity/tfsec-pr-commenter-action action to v1.3.1
• Update checkmarx/kics-action action to v1.6
• Update helm/chart-releaser-action action to v1.6.0
• Update module github.com/aws/aws-sdk-go to v1.54.1
• Update module github.com/prometheus/client_golang to v1.19.1
• Update module github.com/tidwall/gjson to v1.17.1
• Update Terraform terraform-aws-modules/iam/aws to v5
• Update actions/checkout action to v4
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

First of all, great project!

I just wanted to ask you what was the rationale to choose cpuMemRelation = 7.2, which somehow ends up with memory having "bigger weight" o the price.

I was wandering why not assign half of the price to mem and half to cpu

MemPrice = ( p.Price / 2 ) / float64(p.GetMemory())
CPUPrice = ( p.Price / 2 ) / float64(p.GetCPU())

Thanks!

I get this error when I want to deploy with helm on my k8s cluster and pod crashloopbackoff:

panic: NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Can anyone help me?