Issue with Contour after upgrading Kubernetes to 1.8.4 about contour HOT 6 CLOSED

@cdelpinogeo Do you have the exact error you're seeing?

from contour.

Hi @cdelpinogeo

Thanks for raising this issue. I hope I don't sound like I'm passing the buck but this smells like a kubernetes bug. I say this because there's nothing special about contour, it's just a regular old deployment.

I'd try

kubectl delete -f https://j.hept.io/contour-deployment-rbac

Then

kubectl get ns -w

and wait for the heptio-contour namespace to disappear then

kubectl create -f https://j.hept.io/contour-deployment-rbac

If that doesn't seem to fix the problem, can you please paste the output of

kubectl get nodes

Thanks

Dave

from contour.

@timothysc, I've attached a screenshot of the error:

There are no logged messages in the container:

Thanks!

from contour.

@davecheney, thanks for following up. I've tried the steps you mentioned, but I get the same result. Here are the nodes:

NAME                             STATUS    ROLES     AGE       VERSION
ip-172-20-100-83.ec2.internal    Ready     node      6h        v1.8.4
ip-172-20-117-169.ec2.internal   Ready     master    6h        v1.8.4
ip-172-20-125-118.ec2.internal   Ready     node      6h        v1.8.4
ip-172-20-43-164.ec2.internal    Ready     node      6h        v1.8.4
ip-172-20-47-36.ec2.internal     Ready     node      6h        v1.8.4
ip-172-20-50-158.ec2.internal    Ready     master    7h        v1.8.4
ip-172-20-74-166.ec2.internal    Ready     node      6h        v1.8.4
ip-172-20-85-161.ec2.internal    Ready     master    6h        v1.8.4
ip-172-20-90-216.ec2.internal    Ready     node      6h        v1.8.4

Thanks again!

from contour.

@cdelpinogeo thanks for this information. Sadly I don't think I can assist you, that error is coming from the pod scheduler. Somehow your kubernetes thinks that node is still alive, even though it is not listed in get nodes.

from contour.

Thanks for looking into @davecheney. I decided to restart the masters, which fixed the issue. I'll raise this with the kops sig to see if this is an issue. Thanks!

from contour.