The main goal of Nova-kubernetes-drain is to perform live-evacuation of OpenStack compute node when Kubernetes node is being drained. Nova-kubernetes-drain should be deployed as a Daemonset via Stackanetes.
Nova-kubernetes-drain can be run as a daemon or as one-off task. Those two modes are simple configure by command line flag.
Nova-kubernetes-drain is based on the following clients:
- Kubernetes client: https://godoc.org/k8s.io/kubernetes/pkg/client/unversioned
- OpenStack client*: http://gophercloud.io/docs/compute/
* Rackspace client is deprecated, but the new client currently does not support live-migration. The Client should be switched when the client supports live-migration.
Nova-kubernetes-drain requires a configuration file, by default named config.yaml. Configuration file should contain all variables necessary to establish connection with openstack. config.yaml example:
IdentityEndpoint: "http://keystone-api:5000/v3/"
Username: "admin"
Password: "mysupersecretpassword"
TenantName: "admin"
DomainID: "default"
Run once is the default mode. To perform an evacuation, simply run the application without any additional flags. Flag -config-path
is optional. Once the evacuation is successful, the application will exit without any error code.
To run it:
./drainer -config-path=<configuration-file-name>
The application will execute following actions:
- Get authorization data from configuration file.
- Determine the name of the running hypervisor in OpenStack.
- Disable scheduling of VMs on this node in OpenStack.
- Identify all VMs on this node.
- Trigger a live-migration.
- Exit the application if all VMs are migrated or timeout is reached.
To run Nova-kubernetes-drain as a daemon. One has to pass additional -daemon
flag.
./drainer -daemon -config-path=<configuration-file-name>
In this mode, the application will wait for specific Kubernetes events to take actions. A Kubernetes drain operation will disable scheduling of new VMs and perform live-evacuation of the currently running VMs. On the other hand, a Kubernetes uncordon operation will re-enable the scheduling.
Lifecycle of the application:
- Load Openstack authorization variables from file.
- Hook to Kubernetes event stream and wait for proper events.
- According to the event message, trigger the appropriate operation:
- Unschedulable event received:
- Disable nova-compute in Openstack.
- Identify all VMs on this nova-compute node.
- Trigger live-migration for each of those VMs.
- Schedulable event received:
- Enable nova-compute in Openstack.
- Unschedulable event received: