Check if Image exists about krane HOT 3 CLOSED

I agree that it would be great to have this kind of check, but that might be problematic because for security reasons the host that runs kubernetes-deploy might not have access to the container registry. Also the registry auth may be specific to the project (VPN or token).
Maybe we should leave it up to users?

from krane.

I agree with Kir that we shouldn't really assume the deploy host has access to the registry. Conversely, it would be possible (if stranger) for the deploy host to have access but the production cluster not to. We should improve our handling of the situation though for sure.

Did you see warnings logged about the ImangePullBackoff? This code is supposed to log, giving you the chance to realize it is unrecoverable and abort. Minimally we could improve that message to suggest action.

#54 proposes aborting the deploy at some point based on observing this condition. We'd have to think carefully about when we make the "it's doomed" call though... Last week we saw a case where issues with the production env's link to the registry caused a huge deployment to roll out super slowly, with many (but not all) containers in ImagePullBackoff. Ideally that case wouldn't fail the deploy. Maybe we could flag this condition to the parent deployment object, which could make the call when all its children are persistently in that state... still a timeout, but hopefully smarter?

On the other hand, maybe we should keep the log-warning approach and expect deployments to specify progressDeadlineSeonds to actually fail the deploy, for real on the k8s side, not just from our gem's perspective. AFAICT that field seems to be the Kubernetes answer to permanent failure conditions, including this one.

from krane.

IMO #116 resolved this. We now inspect all pods in the new ReplicaSet, and if they are all failing for a recognized reason (including because of failure to pull image), the deploy will fail immediately. Please reopen if you disagree.

from krane.