failure during uninstall about cpu-manager-for-kubernetes HOT 5 OPEN

I did some more digging, and it looks like the "cmk uninstall" command will only work on one node, since it tries to delete the webhook and that can only succeed on one node (since on the other nodes it'll already be deleted). When that fails on the other nodes it aborts the uninstall leaving the node in a partially-installed state.

from cpu-manager-for-kubernetes.

@cbf123 thanks for raising this issue. Are you using the cmk-uninstall pod or the cmk-uninstall daemonset?

from cpu-manager-for-kubernetes.

In the uninstall module all "delete something" functions do sys.exit(1) whenever any exception is raised. I think it would be better to display warnings in the logs and continue trying to remove all resources in the best effort mode without exiting early.

from cpu-manager-for-kubernetes.

I created a draft pull request linked above, which changes uninstall module behaviour to described. Feel free to give it a try and report your feedback.

I marked it as a "Draft" as most of the uninstall related tests fail now, I'll be happy to fix them once we agree that this is the right way to perform the uninstall process.

from cpu-manager-for-kubernetes.

lmdaly: I tried both options. With the cmk-uninstall pod you need to run it on each node manually, and it'll only successfully run on the first node. With the daemonset the first pod to run will be successful, and the other pods fail and keep restarting.

przemeklal: That's one way to do it, and I think it makes sense to clean up as much as we can on an uninstall. I took an alternate approach as attached which is a bit more narrow but I think I like yours better.
diff.txt

from cpu-manager-for-kubernetes.