cordoning node removes the node from loadbalancer causing downtime about eks-rolling-update HOT 4 CLOSED

Are you able to update the description to supply more context about the specific load balancer/health check setup is required to cause this behaviour in an EKS context?

Personally, in my setup, we don't observe this behaviour while using the tool; but we use external NLBs with basic TCP checks on NodePort ingress Services for the ingress so perhaps that is where things differ. Does this require use of LoadBalancer Services to cause an issue?

This will help provide more useful feedback on the PR, especially regarding documentation that might indicate to users which strategy is the right one for them to use.

from eks-rolling-update.

I observed this issue when dealing with ELB as the service controller removes the cordoned nodes from loadbalancer. However, this may not cause downtime as the script scales up ASG before cordoning the nodes and routes to pods are updated on all the new nodes.

from eks-rolling-update.

We experienced this same issue. It's especially acute if you use RUN_MODE=2 and clusterTrafficPolicy: Local on your services. You sometimes get lucky since traffic gets routed to all nodes if none of the nodes are available. But, if you have a lot of traffic and you only have one node in service you can saturate that node very quickly.

It's much better to use run modes that only cordon AZ at a time.

The proposed fix seems like a good one though, so I'm for some variant of #49 being merged.

from eks-rolling-update.

Resolved by #49

from eks-rolling-update.