ocean staging notebooks won't launch about pangeo-cloud-federation HOT 5 CLOSED

In trying to fix this, I managed to make things much worse.

I did the very dumb thing of trying to resize the default pool down to size 0 to kill the messed up nodes.

This operation is stuck in a pending state. As long as it can't complete, I can't try anything else. In the meantime, key pods like autohttps can't get scheduled.

from pangeo-cloud-federation.

So the pending state is resolved and the cluster is superficially back to normal.

However, I am still seeing these errors:

2019-03-24 13:38:44+00:00 [Warning] 0/2 nodes are available: 2 Insufficient cpu, 2 Insufficient memory.
2019-03-24 13:39:00+00:00 [Normal] pod didn't trigger scale-up (it wouldn't fit if a new node is added)

I the n1-standard-highmem node pool should be able to scale up to fit these pods, but for some reason this is not happening.

from pangeo-cloud-federation.

Latest update: I manually scaled the n1-standard-highmem up to size 1, and my notebook finally launched again.

So why would it not scale up on its own?

from pangeo-cloud-federation.

Ok, now the node pool seems to be scaling normally.

from pangeo-cloud-federation.

@rabernat - okay to close?

from pangeo-cloud-federation.