Timeout errors in MutatingWebhookConfiguration about vault-k8s HOT 8 CLOSED

A firewall in gcp was blocking the request. It worked when I allowed traffic to port 8080 from the kubernetes master nodes (172.16.0.0/28).
Thanks again for your time:)

from vault-k8s.

A firewall in gcp was blocking the request. It worked when I allowed traffic to port 8080 from the kubernetes master nodes (172.16.0.0/28).
Thanks again for your time:)

this one comment saved me tons of time ... using GKE private clusters ... had to create a firewall rule so control plane can communicate with pods directly ... essentially opening up 8080 from source "master address range" otherwise the mutatingwebhookconfiguration prevents all new pods from starting (configerror) because the control plane timeouts on trying to communicate with agent injector pod.

thanks and cheers

from vault-k8s.

Hi @krep-dr, are you using istio?

from vault-k8s.

@jasonodonnell Thanks for looking into this. No I don’t use istio. Will the injector log when receiving a request from the webhook?

from vault-k8s.

It will, yes. Is the injector running in the vault namespace? The deploy scripts sets it to that namespace by default.

from vault-k8s.

Yes it is

from vault-k8s.

As the other comments suggested the helm chart doesn't know if you have a private GKE cluster.
Enable communication between the control plane to the nodes.

Here's an example in terraform.

resource "google_compute_firewall" "gke-master-to-node" {
  name    = "gke-master-to-node"
  project = "{YOUR_PROJECT_ID}"
  network = "{YOUR_COMPUTE_NETWORK_ID}"
  allow {
    protocol = "all"
   # all ports exposed or ["443"]  
  }
  source_ranges = ["{MASTER_IPV4_CIDR_BLOCK}"] # e.g "10.1.0.0/28"
  target_tags = ["gke-node"] # nodes must be tagged
}

from vault-k8s.

I had the same problem. I configured group security in aws and now it's working..

from vault-k8s.