max_replicas_per_node in paused/drained nodes drives to -> no suitable node when updating service about classicswarm HOT 9 CLOSED

is the service configured "stop-first" or "start-first"?

from classicswarm.

is the service configured "stop-first" or "start-first"?

Everything is stop-first

from classicswarm.

@sebastianfelipe I would say that it is by design.

You should either:

• reduce number of replicas to 4
• add one more worker so there is always one spare worker.
• schedule maintenance work on time when there is no updates on-going.

from classicswarm.

@sebastianfelipe I would say that it is by design.

You should either:

• reduce number of replicas to 4
• add one more worker so there is always one spare worker.
• schedule maintenance work on time when there is no updates on-going.

It cannot be like that, because:

• if I reduce the number of replicas to 4, it is 2/2 and 2/2 in each worker, so the case is the same
• add one more worker -> cost
• It needs to work with CI/CD effectively, no applying some patch to a bad behaviour that's not clean to me, there's a stack with more than 10 services, so in each docker compose change I'll have to patch everything and that is not what I'm looking for. If I have 6 replicas running and I decrease to 3, what will happen with the performance? Is it necessary to have in mind that the solution cannot be on my hand, it is necessary that every change still working without major problem.

from classicswarm.

Adding to this, it is a service with stop-first, so in theory, the max_replicas_per_node, should work.

from classicswarm.

Well this kind of use case didn't came to my mind when I implemented max replicas feature. We use it just make sure that all replicas do not get scheduled to one worker when another one goes down e.g. in case of virtualization platform crash, network failure or node reboot.

So this need to be handled on swarmkit side on way that someone implements test case which handles this situation and then modify logic to handle it correctly.

PS. IMO patching existing workers is very old skool approach. If you create flow which first add new already patched worker to swarm and then drop old one you will not see this issue.

from classicswarm.

@olljanat thanks for answering. I didn't understand, so you're saying that you thought in the case a VM is down, so what did happen here? Because I turned off definitely a VM and then appears that error, so the question is, why is it pretending to deploy services on down hosts? Maybe I didn't understand your point very well :/

from classicswarm.

@sebastianfelipe I mean if you deploy service with two replicas to swarm where is two workers both of them will run one replica. If you now reboot one of those workers then swarm will notice that there is only one running replica service and schedule second replica running on that only available worker so end result is that one worker have two replicas of service running and second worker have zero replicas.

Only way to fix that without max replicas is scale all services to 1 and back to 2 after worker reboot. Other why you end up situation where you application is not anymore fault tolerant for worker crash (if worker which have all replicas goes down then whole application is down until swarm re-start new replicas to another worker).

Btw. You see how this was implemented and which kind of tests was included from moby/swarmkit#2758 and that it what is needed to enhance to handle your use case correctly.

from classicswarm.

This repo is not for Docker swarm you are looking for https://github.com/docker/swarmkit

from classicswarm.