darxkies / k8s-tew Goto Github PK

Kubernetes - The Easier Way

License: GNU General Public License v3.0

Go 95.28% Makefile 2.30% Shell 1.83% Dockerfile 0.59%

kubernetes cluster docker containers cncf containerd ceph helm grafana prometheus

k8s-tew's Introduction

Kubernetes - The Easier Way (k8s-tew)

k8s-tew is a CLI tool to install a Kubernetes Cluster (local, single-node, multi-node or HA-cluster) on Bare Metal. It installs the most essential components needed by a cluster such as networking, storage, monitoring, logging, backuping/restoring and so on. Besides that, k8s-tew is also a supervisor that starts all cluster components on each node, once it setup the nodes.

TL;DR

Documentation

The project documentation can be found here: https://darxkies.github.io/k8s-tew

Caveats

The local setup uses for ingress the ports 80, 443 so they need to be free on the host. It also turns swapping off which is a requirement for kubelet.
On CentOS nodes the firewall and SELinux are disabled to not interfere with Kubernetes.

Feedback

E-Mail: [email protected]

k8s-tew's People

Contributors

Stargazers

Watchers

k8s-tew's Issues

CephFS Stroage Class needs fsName field

Otherwise you get a missing field fsName when provisioning. Correct definition is:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  namespace: storage
  name: csi-cephfs
provisioner: cephfs.csi.ceph.com
reclaimPolicy: Delete
parameters:
  clusterID: clusterid
  fsName: cephfs
  pool: cephfs
  mounter: kernel
  provisionVolume: "true"
  csi.storage.k8s.io/provisioner-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/provisioner-secret-namespace: storage
  csi.storage.k8s.io/controller-expand-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: storage
  csi.storage.k8s.io/node-stage-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/node-stage-secret-namespace: storage
  allowVolumeExpansion: "true"

Install k8s-tew without updating environment or running as a service

I went through the install and usage instructions, only to find that KUBECONFIG was being set in /etc/environment. Also noticed k8s-tew creates a systemd service where its installed. I find this a bit intrusive since no other Kubernetes commands/apps do such a thing. Its too forceful. Makes assumptions this is what I wanted. I do not see a reason why this cannot be a free standing cli that can be executed from anywhere as long as its in the user's PATH.

As is, I cannot use this anywhere other than a homelab since it makes too many undesirable changes to the system. No other Kubernetes cli tools does this: kubectl, istioctl, etc, and all manage to work with Kube clusters just fine. I cannot raise a PR since I do not know if these changes are absolute requirements for k8s-tew to work. If so, that would be unfortunate.

How can we use istio sidecar and its loadbalancers

Where does istio fit in here, is there a way to configure it and use istio as sidecar and loadbalncer.

Also anyway no to install Wordpress and instead any other image?

restic and fluent-bit fail to start on controller nodes due to cni plugin not initialized

Commands run:

k8s-tew node-add -n node0-i 192.168.10.XX -x 0 -l controller
k8s-tew node-add -n node1 -i 192.168.10.XX -x 1 -l controller
k8s-tew node-add -n node2 -i 192.168.10.XX -x 2 -l controller
k8s-tew node-add -n node3 -i 192.168.10.XX -x 3 -l worker,storage
k8s-tew configure --public-network 192.168.10.0/24
k8s-tew generate --parallel --pull-images
k8s-tew deploy --import-images

Log output for containerd.log:

root@node0:/var/log/k8s-tew# cat containerd.log | grep error
time="2020-09-01T21:20:10.292933160Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/k8s-tew/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2020-09-01T21:20:10.292979721Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2020-09-01T21:20:10.294183989Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/k8s-tew/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2020-09-01T21:20:10.294224005Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2020-09-01T21:20:10.305771036Z" level=error msg="Failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/k8s-tew/cni: cni plugin not initialized: failed to load cni config"

Log output for kubelet.log is full of:

E0901 22:13:21.631931  185025 pod_workers.go:191] Error syncing pod 5c7fa076-8834-41c5-b7b8-90f146eb5bcc ("restic-dsq5t_backup(5c7fa076-8834-41c5-b7b8-90f146eb5bcc)"), skipping: network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized

Using k8s-tew version 2.4.0-beta.5, running Ubuntu 20.04.1 LTS, this is consistent across all 3 controller nodes deployed

Support for MetalLB

What's the process to install additional dependencies to the cluster upon creation?
Example, I'm interested in MetalLB so Services can have External IPs when they get created. At the moment, upon cluster creation (ubuntu-multi) everything is properly created, but the Services do not have External IPs. If I install MetalLB after cluster creation, existing Services do not get External IPs (that may be a limitation of MetalLB, therefore I'm interested in installing that early.

Is there a way to add a SAN to the api server certificate?

I'm running k8s-tew behind a load balancer external to the cluster, but I'm unable to point kubectl to the lb.
My workaround is as follows:

modify kubeconfig.yaml to point to one of the nodes: server: https://node1:16443
at /etc/hosts entry pointing node1 to the load balancer on my local machine

Suffice to say this is not ideal. There are ways to add alternate names on kubeadmn setups. Is there a way using k8s-tew, or maybe this could be something to add as a command?

Thanks!

Unable to reach service from node

The cluster has been working like a charm, but I'm unable to reach cluster services from the node even though I see all the calico interfaces. Seems to be no route from the node to the cluster. Is this a calico configuration issue that can be modified? My use case is in this SO question.

Thanks!

letsencrypt-cluster-issuer-setup step fails to find cert-manager-webhook

Commands run:

k8s-tew node-add -n node0-i 192.168.10.XX -x 0 -l controller
k8s-tew node-add -n node1 -i 192.168.10.XX -x 1 -l controller
k8s-tew node-add -n node2 -i 192.168.10.XX -x 2 -l controller
k8s-tew node-add -n node3 -i 192.168.10.XX -x 3 -l worker,storage
k8s-tew configure --public-network 192.168.10.0/24
k8s-tew configure --email [email protected] --ingress-domain acme.com
k8s-tew generate --parallel --pull-images
k8s-tew deploy --import-images

Deployment output:

...<snip>...
INFO[0262] Applying manifest                             name=kubelet-setup
INFO[0263] Applying manifest                             name=admin-user-setup
INFO[0263] Applying manifest                             name=calico-setup
INFO[0263] Applying manifest                             name=metallb-setup
INFO[0263] Applying manifest                             name=coredns-setup
INFO[0263] Applying manifest                             name=ceph-secrets
INFO[0263] Applying manifest                             name=ceph-manager-credentials
INFO[0263] Applying manifest                             name=ceph-setup
INFO[0263] Applying manifest                             name=ceph-csi
INFO[0264] Applying manifest                             name=kubernetes-dashboard-setup
INFO[0264] Applying manifest                             name=cert-manager-setup
INFO[0264] Applying manifest                             name=nginx-ingress-setup
INFO[0264] Applying manifest                             name=letsencrypt-cluster-issuer-setup
ERRO[0269] Manifest failed                               error="Could not patch '/letsencrypt-production': Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": Post https://cert-manager-webhook.networking.svc:443/mutate?timeout=30s: no endpoints available for service \"cert-manager-webhook\"" manifest=/home/tyler/assets/etc/k8s-tew/k8s/setup/letsencrypt-cluster-issuer.yaml name=letsencrypt-cluster-issuer-setup
ERRO[0269] Failed deploying                              error="Could not patch '/letsencrypt-production': Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": Post https://cert-manager-webhook.networking.svc:443/mutate?timeout=30s: no endpoints available for service \"cert-manager-webhook\""

I don't see anything in the cert-manager namespace:

tyler@TYLER-DESKTOP:~$ kubectl get ns
NAME              STATUS   AGE
backup            Active   7h2m
cert-manager      Active   7h2m
default           Active   7h13m
kube-node-lease   Active   7h13m
kube-public       Active   7h13m
kube-system       Active   7h13m
logging           Active   7h2m
monitoring        Active   7h2m
networking        Active   7h3m
showcase          Active   7h2m
storage           Active   7h3m
tyler@TYLER-DESKTOP:~$ kubectl get all -n cert-manager
No resources found in cert-manager namespace.

Digging into the networking namespace and I seem to find what it's looking for (specifically the cert-manager-webhook service):

tyler@TYLER-DESKTOP:~$ kubectl get all -n networking
NAME                                               READY   STATUS             RESTARTS   AGE
pod/calico-kube-controllers-7b6d6f655c-b9fgd       1/1     Running            0          7h3m
pod/calico-node-hgg6j                              0/1     CrashLoopBackOff   7          7h3m
pod/calico-node-hszzt                              1/1     Running            0          7h3m
pod/calico-node-rrwjz                              0/1     CrashLoopBackOff   7          7h3m
pod/calico-node-zqdtc                              0/1     CrashLoopBackOff   7          7h3m
pod/calico-typha-94b899f97-zjdpb                   1/1     Running            0          7h3m
pod/cert-manager-6cd74c57c6-26nxd                  1/1     Running            0          7h3m
pod/cert-manager-cainjector-b9d7f4ddf-9rlp9        1/1     Running            0          7h3m
pod/cert-manager-webhook-7cbdbb9fc-sjcc9           1/1     Running            0          7h3m
pod/metallb-controller-8457fb8fb5-qdvsd            1/1     Running            0          7h3m
pod/metallb-speaker-bm4hw                          1/1     Running            0          7h3m
pod/nginx-ingress-controller-bdf4cdd7b-v4m89       1/1     Running            0          7h3m
pod/nginx-ingress-default-backend-89668fbd-fw2f9   1/1     Running            0          7h3m

NAME                                    TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                      AGE
service/calico-typha                    ClusterIP      10.32.0.5     <none>         5473/TCP                     7h3m
service/cert-manager                    ClusterIP      10.32.0.151   <none>         9402/TCP                     7h3m
service/cert-manager-webhook            ClusterIP      10.32.0.130   <none>         443/TCP                      7h3m
service/ingress-nginx                   LoadBalancer   10.32.0.23    192.168.0.16   80:31672/TCP,443:32098/TCP   7h3m
service/nginx-ingress-default-backend   ClusterIP      10.32.0.97    <none>         80/TCP                       7h3m

NAME                             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/calico-node       4         4         1       4            1           kubernetes.io/os=linux   7h3m
daemonset.apps/metallb-speaker   1         1         1       1            1           <none>                   7h3m

NAME                                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/calico-kube-controllers         1/1     1            1           7h3m
deployment.apps/calico-typha                    1/1     1            1           7h3m
deployment.apps/cert-manager                    1/1     1            1           7h3m
deployment.apps/cert-manager-cainjector         1/1     1            1           7h3m
deployment.apps/cert-manager-webhook            1/1     1            1           7h3m
deployment.apps/metallb-controller              1/1     1            1           7h3m
deployment.apps/nginx-ingress-controller        1/1     1            1           7h3m
deployment.apps/nginx-ingress-default-backend   1/1     1            1           7h3m

NAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/calico-kube-controllers-7b6d6f655c       1         1         1       7h3m
replicaset.apps/calico-typha-94b899f97                   1         1         1       7h3m
replicaset.apps/cert-manager-6cd74c57c6                  1         1         1       7h3m
replicaset.apps/cert-manager-cainjector-b9d7f4ddf        1         1         1       7h3m
replicaset.apps/cert-manager-webhook-7cbdbb9fc           1         1         1       7h3m
replicaset.apps/metallb-controller-8457fb8fb5            1         1         1       7h3m
replicaset.apps/nginx-ingress-controller-bdf4cdd7b       1         1         1       7h3m
replicaset.apps/nginx-ingress-default-backend-89668fbd   1         1         1       7h3m

I also tried applying the manifest (assets/etc/k8s-tew/k8s/setup/cert-manager-setup.yaml ) manually but it only applied some of the resources due to more than one namespace being provided:

tyler@TYLER-DESKTOP:~$ kubectl apply -n cert-manager -f /home/tyler/assets/etc/k8s-tew/k8s/setup/cert-manager-setup.yaml
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/cert-manager configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:auth-delegator configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:webhook-requester configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-view configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/cert-manager-edit configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' to perform this operation.
the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' to perform this operation.
the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' to perform this operation.
the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' to perform this operation.
the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.
the namespace from the provided object "networking" does not match the namespace "cert-manager". You must pass '--namespace=networking' to perform this operation.

Using k8s-tew version 2.4.0-beta.5, running Ubuntu 20.04.1 LTS

k8s-tew run failed

Hi mate,

I was running k8s-tew run and it failed with the below information.

[ERRO[0326] Restarting server command="/home/jim/assets/opt/k8s-tew/bin/k8s/kube-apiserver --bind-address=0.0.0.0 --kubelet-client-certificate=/home/jim/assets/etc/k8s-tew/ssl/kubernetes.pem --proxy-client-key-file=/home/jim/assets/etc/k8s-tew/ssl/aggregator-key.pem --requestheader-allowed-names=aggregator,admin,system:kube-controller-manager,system:kube-controller-manager,system:kube-scheduler,system:node:centos74.master --tls-private-key-file=/home/jim/assets/etc/k8s-tew/ssl/kubernetes-key.pem --apiserver-count=2 --etcd-keyfile=/home/jim/assets/etc/k8s-tew/ssl/kubernetes-key.pem --kubelet-certificate-authority=/home/jim/assets/etc/k8s-tew/ssl/ca.pem --advertise-address=192.168.4.222 --allow-privileged=true --audit-log-path=/home/jim/assets/var/log/k8s-tew/audit.log --enable-admission-plugins=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota --etcd-cafile=/home/jim/assets/etc/k8s-tew/ssl/ca.pem --feature-gates=KubeletPluginsWatcher=true,CSIBlockVolume=true,BlockVolume=true --requestheader-group-headers=X-Remote-Group --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --enable-swagger-ui=true --requestheader-client-ca-file=/home/jim/assets/etc/k8s-tew/ssl/ca.pem --service-node-port-range=30000-32767 --tls-cert-file=/home/jim/ERRO[0329] Command failed command="systemctl disable firewalld && systemctl stop firewalld && setenforce 0 && sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux" error="Command 'systemctl disable firewalld && systemctl stop firewalld && setenforce 0 && sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux' failed with error 'exit status 1' (Output: setenforce: SELinux is disabled\n)" name=setup-centos [/] PrFATA[0329] Cluster setup failed error="Command 'systemctl disable firewalld && systemctl stop firewalld && setenforce 0 && sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux' failed with error 'exit status 1' (Output: setenforce: SELinux is disabled](url)

Not sure what is the reason but I did restart the server.

OS=CentOS 7.5 installed on Windows 10 Enterprise, Virtualisation Software Virtualbox

Kind Regards,
Ameya Agashe

exporting PATH as part of k8s-tew environment needs double quotes

Hi,

I ran eval $(k8s-tew environment) and it would set some variables but not all in my environment. anything after exporting PATH would not make it, I dug into it and its because of a syntax error related to the existing PATH:

bash: test.sh: line 22445: syntax error near unexpected token `('
bash: test.sh: line 22445: `export PATH=/home/tyler/assets/opt/k8s-tew/bin:/home/tyler/assets/opt/k8s-tew/bin/k8s:/home/tyler/assets/opt/k8s-tew/bin/etcd:/home/tyler/assets/opt/k8s-tew/bin/cri:/home/tyler/assets/opt/k8s-tew/bin/cni:/home/tyler/assets/opt/k8s-tew/bin/velero:/home/tyler/assets/opt/k8s-tew/bin/host:/home/tyler/bin:/home/tyler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/python/scripts/:/mnt/c/python/:/mnt/c/program files (x86)/common files/oracle/java/javapath:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/system32/wbem:/mnt/c/windows/system32/windowspowershell/v1.0/:/mnt/c/windows/system32/openssh/:/mnt/c/program files/calibre2/:/mnt/c/program files/microsoft vs code/bin:/mnt/c/users/tyler/appdata/roaming/cloud foundry:/mnt/c/Program Files/FileBot/:/mnt/c/Users/tyler/AppData/Local/Microsoft/WindowsApps:/mnt/c/Program Files (x86)/Nmap:/snap/bin:/usr/local/go/bin:/home/tyler/projects/go/bin'

The export chokes on the ( character because the export isn't wrapped in double quotes, if you wrap it in double quotes like so:

export PATH="/home/tyler/assets/opt/k8s-tew/bin:/home/tyler/assets/opt/k8s-tew/bin/k8s:/home/tyler/assets/opt/k8s-tew/bin/etcd:/home/tyler/assets/opt/k8s-tew/bin/cri:/home/tyler/assets/opt/k8s-tew/bin/cni:/home/tyler/assets/opt/k8s-tew/bin/velero:/home/tyler/assets/opt/k8s-tew/bin/host:/home/tyler/bin:/home/tyler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/python/scripts/:/mnt/c/python/:/mnt/c/program files (x86)/common files/oracle/java/javapath:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/system32/wbem:/mnt/c/windows/system32/windowspowershell/v1.0/:/mnt/c/windows/system32/openssh/:/mnt/c/program files/calibre2/:/mnt/c/program files/microsoft vs code/bin:/mnt/c/users/tyler/appdata/roaming/cloud foundry:/mnt/c/Program Files/FileBot/:/mnt/c/Users/tyler/AppData/Local/Microsoft/WindowsApps:/mnt/c/Program Files (x86)/Nmap:/snap/bin:/usr/local/go/bin:/home/tyler/projects/go/bin"

Without double quoting PATH:

tyler@TYLER-DESKTOP:~$ eval $(k8s-tew environment)
tyler@TYLER-DESKTOP:~$ echo $KUBECONFIG

tyler@TYLER-DESKTOP:~$ k8s-tew environment > test.sh
tyler@TYLER-DESKTOP:~$ . test.sh
bash: test.sh: line 22445: syntax error near unexpected token `('
bash: test.sh: line 22445: `export PATH=/home/tyler/assets/opt/k8s-tew/bin:/home/tyler/assets/opt/k8s-tew/bin/k8s:/home/tyler/assets/opt/k8s-tew/bin/etcd:/home/tyler/assets/opt/k8s-tew/bin/cri:/home/tyler/assets/opt/k8s-tew/bin/cni:/home/tyler/assets/opt/k8s-tew/bin/velero:/home/tyler/assets/opt/k8s-tew/bin/host:/home/tyler/bin:/home/tyler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/python/scripts/:/mnt/c/python/:/mnt/c/program files (x86)/common files/oracle/java/javapath:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/system32/wbem:/mnt/c/windows/system32/windowspowershell/v1.0/:/mnt/c/windows/system32/openssh/:/mnt/c/program files/calibre2/:/mnt/c/program files/microsoft vs code/bin:/mnt/c/users/tyler/appdata/roaming/cloud foundry:/mnt/c/Program Files/FileBot/:/mnt/c/Users/tyler/AppData/Local/Microsoft/WindowsApps:/mnt/c/Program Files (x86)/Nmap:/snap/bin:/usr/local/go/bin:/home/tyler/projects/go/bin'

With double quoting PATH:

tyler@TYLER-DESKTOP:~$ nano test.sh
tyler@TYLER-DESKTOP:~$ . test.sh
tyler@TYLER-DESKTOP:~$ echo $KUBECONFIG
/home/tyler/assets/etc/k8s-tew/k8s/kubeconfig/admin.kubeconfig

This is most likely due to me running the command on WSL (thus the Program Files (x86)), but it would also prevent it from happening to any other users that happen to have parenthesis in their PATH

Using k8s-tew version 2.4.0-beta.5, running Ubuntu 20.04.1 LTS on nodes

Coredns is starving resources in single node setup on ubuntu 18.04

Hi,

When deploying k8s-tew 2.2.4 in standalone mode, coredns is consuming all CPU resources of the machine.

How to reproduce :

deploy a machine on ubuntu 18.04
follow quickstart guide
display htop and see "/coredns -conf /etc/coredns/Corefile" consume 330-500% CPU.

When killing coredns with -9 flag, all server is calming down, but of course, dns is not working anymore in cluster !

Add Twitter and Linkedin page at Readme.md

Description
Currently, our project's README.md lacks links to our official Twitter and LinkedIn pages. To enhance our project's online presence and make it easier for users to connect with us, let's add these social media links to the README.

Proposed Changes
Create a new section in the README.md file titled "Social Media."
Add subsections for Twitter and LinkedIn.
Insert hyperlinks to our official Twitter and LinkedIn pages.

Benefits
Improve project visibility and engagement.
Facilitate communication with users and the community.
Showcase our project on multiple platforms.

Additional Notes
Make sure to replace placeholders like @YourTwitterHandle and Your LinkedIn Page with the actual handles or names.
Verify the correctness of the provided hyperlinks.

k8s-tew.service is not getting installed, which causes deployments to fail

Here are my steps:

# Installation
curl -s https://api.github.com/repos/darxkies/k8s-tew/releases/latest | grep "browser_download_url" | cut -d : -f 2,3 | tr -d \" | sudo wget -O /usr/local/bin/k8s-tew -qi -

sudo chmod a+x /usr/local/bin/k8s-tew

# Initialization and Configuration
k8s-tew initialize
k8s-tew node-add -n controller00 -i a.b.c.d -x 0 -l bootstrapper,controller,worker
k8s-tew generate
k8s-tew deploy -i ~/.ssh/id_rsa

Everything runs and then I get:

INFO[0009] Executing remote command                      name=start-service node=controller00
ERRO[0011] Failed deploying                              error="Process exited with status 1"

So I ran it again with k8s-tew deploy -i ~/.ssh/id_rsa -d and I saw it was trying to load/start/start the k8s-tew service in the remote, but that service is never installed.

If I run systemctl enable k8s-tew, I get
Failed to enable unit: Unit file k8s-tew.service does not exist.

No matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1alpha1"

I successfully installed a cluster on a previous machine, but I'm now trying with a different cluster (same machine type/specs with ubuntu 18.04), and can't get past the ceph-csi command during deploy, which returns the error:

unable to recognize "/home/user/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1alpha1"

Typo in Generate

https://github.com/darxkies/k8s-tew/search?q=kubenetes&unscoped_q=kubenetes

ERRO[0456] Cluster setup failed

I am trying to do a single node install on ubuntu 18.04 (local bare-metal). I get this error when I try to execute run. any idea what the issue is?
here is the output for node-list (I used defaults for config):
k8s-tew node-list
INFO[0000] Node index=0 ip=10.200.26.80 labels="[bootstrapper controller worker]" name=gpx10

ERRO[0456] Command failed command="/home/user/k8-tew/assets/opt/k8s-tew/bin/k8s/kubectl --request-timeout 30s --kubeconfig /home/user/k8-tew/assets/etc/k8s-tew/k8s/kubeconfig/admin.kubeconfig apply -f /home/user/k8-tew/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml" error="Command '/home/user/k8-tew/assets/opt/k8s-tew/bin/k8s/kubectl --request-timeout 30s --kubeconfig /home/user/k8-tew/assets/etc/k8s-tew/k8s/kubeconfig/admin.kubeconfig apply -f /home/user/k8-tew/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml' failed with error 'exit status 1' (Output: storageclass.storage.k8s.io/csi-rbd unchanged\nstorageclass.storage.k8s.io/csi-cephfs unchanged\nserviceaccount/cephfs-csi-attacher unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-external-attacher-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-attacher-role unchanged\nservice/csi-cephfsplugin-attacher unchanged\nstatefulset.apps/csi-cephfsplugin-attacher unchanged\nservice/csi-cephfsplugin-provisioner unchanged\nstatefulset.apps/csi-cephfsplugin-provisioner unchanged\ndaemonset.apps/csi-cephfsplugin unchanged\nserviceaccount/cephfs-csi-nodeplugin unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-csi-nodeplugin unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-nodeplugin unchanged\nserviceaccount/cephfs-csi-provisioner unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-external-provisioner-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-provisioner-role unchanged\nrole.rbac.authorization.k8s.io/cephfs-external-provisioner-cfg unchanged\nrolebinding.rbac.authorization.k8s.io/cephfs-csi-provisioner-role-cfg unchanged\nserviceaccount/rbd-csi-attacher unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-external-attacher-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-attacher-role unchanged\nserviceaccount/rbd-csi-nodeplugin unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-csi-nodeplugin unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-nodeplugin unchanged\nserviceaccount/rbd-csi-provisioner unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-external-provisioner-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-provisioner-role unchanged\nrole.rbac.authorization.k8s.io/rbd-external-provisioner-cfg unchanged\nrolebinding.rbac.authorization.k8s.io/rbd-csi-provisioner-role-cfg unchanged\nservice/csi-rbdplugin-attacher unchanged\nstatefulset.apps/csi-rbdplugin-attacher unchanged\nservice/csi-rbdplugin-provisioner unchanged\nstatefulset.apps/csi-rbdplugin-provisioner unchanged\ndaemonset.apps/csi-rbdplugin unchanged\nerror: unable to recognize "/home/user/k8-tew/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1alpha1"\n)" name=ceph-csi
ERRO[0456] Cluster setup failed error="Command '/home/user/k8-tew/assets/opt/k8s-tew/bin/k8s/kubectl --request-timeout 30s --kubeconfig /home/user/k8-tew/assets/etc/k8s-tew/k8s/kubeconfig/admin.kubeconfig apply -f /home/user/k8-tew/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml' failed with error 'exit status 1' (Output: storageclass.storage.k8s.io/csi-rbd unchanged\nstorageclass.storage.k8s.io/csi-cephfs unchanged\nserviceaccount/cephfs-csi-attacher unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-external-attacher-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-attacher-role unchanged\nservice/csi-cephfsplugin-attacher unchanged\nstatefulset.apps/csi-cephfsplugin-attacher unchanged\nservice/csi-cephfsplugin-provisioner unchanged\nstatefulset.apps/csi-cephfsplugin-provisioner unchanged\ndaemonset.apps/csi-cephfsplugin unchanged\nserviceaccount/cephfs-csi-nodeplugin unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-csi-nodeplugin unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-nodeplugin unchanged\nserviceaccount/cephfs-csi-provisioner unchanged\nclusterrole.rbac.authorization.k8s.io/cephfs-external-provisioner-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cephfs-csi-provisioner-role unchanged\nrole.rbac.authorization.k8s.io/cephfs-external-provisioner-cfg unchanged\nrolebinding.rbac.authorization.k8s.io/cephfs-csi-provisioner-role-cfg unchanged\nserviceaccount/rbd-csi-attacher unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-external-attacher-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-attacher-role unchanged\nserviceaccount/rbd-csi-nodeplugin unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-csi-nodeplugin unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-nodeplugin unchanged\nserviceaccount/rbd-csi-provisioner unchanged\nclusterrole.rbac.authorization.k8s.io/rbd-external-provisioner-runner unchanged\nclusterrolebinding.rbac.authorization.k8s.io/rbd-csi-provisioner-role unchanged\nrole.rbac.authorization.k8s.io/rbd-external-provisioner-cfg unchanged\nrolebinding.rbac.authorization.k8s.io/rbd-csi-provisioner-role-cfg unchanged\nservice/csi-rbdplugin-attacher unchanged\nstatefulset.apps/csi-rbdplugin-attacher unchanged\nservice/csi-rbdplugin-provisioner unchanged\nstatefulset.apps/csi-rbdplugin-provisioner unchanged\ndaemonset.apps/csi-rbdplugin unchanged\nerror: unable to recognize "/home/user/k8-tew/assets/etc/k8s-tew/k8s/setup/ceph-csi.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1alpha1"\n)"

Uninstall k8s-tew completly

Recently I've tried the k8s-tew project following the steps described. But now I'd like to completly remove from my system everything. How to proceed? What steps are involved to remove everything?

CentOS 7 - Cannot find package.

# System Info
cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
[..]

# Hypervisor: ESXi 6.5

# Setup: 4 Core, 3GB RAM

# go version
go version go1.8.3 linux/amd64

# git version
git version 1.8.3.1

Problem: Cannot install with go install

[root@k8s-master-01 ~]# go install github.com/darxkies/k8s-tew/cmd/k8s-tew
can't load package: package github.com/darxkies/k8s-tew/cmd/k8s-tew: cannot find package "github.com/darxkies/k8s-tew/cmd/k8s-tew" in any of:
/usr/lib/golang/src/github.com/darxkies/k8s-tew/cmd/k8s-tew (from $GOROOT)
/root/go/src/github.com/darxkies/k8s-tew/cmd/k8s-tew (from $GOPATH)

Binary (https://github.com/darxkies/t8s-tew/releases) ends with 404.

Where can I find the kubeconfig or token to log into the dashboard?

I've started a multi-node cluster (1 controller and 2 workers) successfully.
I can list PODs, nodes, etc.
But for the life of me I cannot figure out where to get the credentials to log into the dashboard.
Usually, I get the kubeconfig from the controller, or look for a ~/.kube/config file but none of those have worked. The last Kube cluster version I've worked in was 1.9 and maybe creds locations have changed since then.

# Commands I've used to spin up a cluster
cd setup/ubuntu-multi-node
make run
eval $(k8s-tew environment)
make dashboard

Fork where I'm tracking my changes: https://github.com/leonj1/k8s-tew/tree/multi_node

Is there a better way to deploy than to upload to nodes via SCP?

Depending on the local connection, it might take a while to deploy to multiple nodes. So it might be faster to generate the artifacts on each node. This also solves the issue of installing k8s-tew on a mac for example, and deploying to another OS like ubuntu.

If you run k8s-tew on a mac, it fails to generate the bash completion files because the generated executables do not run on a mac (kubectl for example). But running each of these directly on the nodes would solve this.

Ceph RBD: missing ID field 'userID' in secrets

When I upgraded to the 2.4.0 version, I kept getting a "failed to provision volume with StorageClass "csi-rbd": rpc error: missing ID field 'userID' in secrets" error when launching pods with that storage class.

It seems that the rbdplugin requires this field in the secrets, so I had to modify the secret to:

apiVersion: v1
kind: Secret
metadata:
    name: csi-rbd-secret
    namespace: storage
data:
    admin: base64(admin_secret)
    k8s-tew: base64(k8s-tew_secret)
    userID: base64(k8s-tew)
    userKey: base64(k8s-tew_secret)

Kill processes on node during deploy

Whenever a deploy fails and a change requires me to run generate, I have to deploy with --force-upload. But if a file on the node is already locked by a process (eg. containerd-shim), then the deployment fails. It would be nice as a preparation to kill these processes before deploying on a node.

error="open ~/.ssh/id_rsa: no such file or directory" on deploy

Hi great concept, I've been wanting to find a lightweight way of deploying.
I'm getting the id_rsa file not found error when running deploy.

ERRO[0000] deploy failed error="open ~/.ssh/id_rsa: no such file or directory"

Have confirmed that $HOME/.ssh/id_rsa is present (on the node I'm running the command from) and am running the command as root.
OS is centos 7
Not sure where I'm going wrong here. I'm running the deploy command from the controller node if that makes any difference.

This is how I've configured the nodes

nodes:
kubemaster01:
ip: 10.1.1.197
index: 0
labels:
- controller
kubenode01:
ip: 10.1.1.198
index: 1
labels:
- worker
kubenode02:
ip: 10.1.1.199
index: 2
labels:
- worker

coredns CrashLoopBackOff Loop detected

Commands run:

k8s-tew node-add -n node0-i 192.168.10.XX -x 0 -l controller
k8s-tew node-add -n node1 -i 192.168.10.XX -x 1 -l controller
k8s-tew node-add -n node2 -i 192.168.10.XX -x 2 -l controller
k8s-tew node-add -n node3 -i 192.168.10.XX -x 3 -l worker,storage
k8s-tew configure --public-network 192.168.10.0/24
k8s-tew generate --parallel --pull-images
k8s-tew deploy --import-images

Log output for coredns pod:

.:53
[INFO] plugin/reload: Running configuration MD5 = 4e235fcc3696966e76816bcd9034ebc7
CoreDNS-1.6.9
linux/amd64, go1.14.1, 1766568
[FATAL] plugin/loop: Loop (127.0.0.1:33564 -> :53) detected for zone ".", see https://coredns.io/plugins/loop#troubleshooting. Query: "HINFO 5265883604060926287.924288037713224908."

Coredns configmap:

{
	"Corefile": ".:53 {
		    errors
		    health {
		        lameduck 5s
		    }
		    ready
		    kubernetes cluster.local in-addr.arpa ip6.arpa {
		      pods insecure
		      fallthrough in-addr.arpa ip6.arpa
		      ttl 30
		    }
		    prometheus :9153
		    forward . /etc/resolv.conf
		    cache 30
		    loop
		    reload
		    loadbalance
		}
		"
}

Using k8s-tew version 2.4.0-beta.5, running Ubuntu 20.04.1 LTS, this is only effecting my worker node

Use etcd as calico data store to allow network policy manager

I'm struggling with installing the calico network policy controller as it requires that the calico node uses etcd as datastore, but it uses kubernetes data store.

As a feature request, is it possible to also add the policy controller during initial setup or via a flag like the other modules?

CNCF conformance test result for k8s-tew with Kubernetes 1.8.5

Output of the CNCF K8S conformance test:

Summarizing 5 Failures:

[Fail] [sig-network] Networking Granular Checks: Pods [It] should function for intra-pod communication: http [Conformance] 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:218

[Fail] [k8s.io] KubeletManagedEtcHosts [It] should test kubelet managed /etc/hosts file [Conformance] 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/exec_util.go:106

[Fail] [sig-network] Networking Granular Checks: Pods [It] should function for intra-pod communication: udp [Conformance] 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:218

[Fail] [sig-network] Networking Granular Checks: Pods [It] should function for node-pod communication: http [Conformance] 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:274

[Fail] [sig-network] Networking Granular Checks: Pods [It] should function for node-pod communication: udp [Conformance] 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:274

Ran 125 of 699 Specs in 3239.982 seconds
FAIL! -- 120 Passed | 5 Failed | 0 Pending | 574 Skipped --- FAIL: TestE2E (3240.02s)

Worked great, Thanks.

error="current host not found in the list of nodes" when trying local installation

Hi,
I'm trying to instal locally on Ubuntu 18.04 and I get the error "ERRO[0001] Failed to run error="current host not found in the list of nodes"". I run the following commands:

k8s-tew initialize
k8s-tew configure --email [email protected] --ingress-domain domain.com
k8s-tew node-add -s
k8s-tew generate --parallel
sudo k8s-tew run

My IP address is in 192.168.1.0/24 but in node-list I see a node with IP address 192.168.100.50.

I tried to go over the documentation but I couldn't figure out what's wrong. Thanks!

how does vagrant setup for k8s-tew to expose to public domain

I followed all instructions and got the server running but not able to connect to server

networking service/ingress-nginx-controller LoadBalancer 10.32.0.60 192.168.120.200 80:32679/TCP,443:32070/TCP 34m

My public IP is different and how to route traffic from public IP to server?

My server is remote and ubuntu bare metal server and can't really access dashboard or anything as no default browser exists.

In my config

public-network: 192.168.120.0/24
...
metallb-addresses: 192.168.120.200/32

here is my ingress config

kubectl get ingress -A
NAMESPACE   NAME                        CLASS    HOSTS                    ADDRESS           PORTS     AGE
showcase    cm-acme-http-solver-24mmx   <none>   wordpress.entrymon.com   192.168.120.200   80        130m
showcase    wordpress                  nginx    wordpress.entrymon.com   192.168.120.200   80, 443   131m

here is the multi node cluster nodes

kubectl get nodes -o wide
NAME           STATUS   ROLES            AGE    VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
controller00   Ready    master           163m   v1.21.5   192.168.120.200   <none>        Ubuntu 20.04.2 LTS   5.4.0-80-generic   containerd://1.5.5
controller01   Ready    master           163m   v1.21.5   192.168.120.201   <none>        Ubuntu 20.04.2 LTS   5.4.0-80-generic   containerd://1.5.5
controller02   Ready    master           163m   v1.21.5   192.168.120.202   <none>        Ubuntu 20.04.2 LTS   5.4.0-80-generic   containerd://1.5.5
worker00       Ready    storage,worker   162m   v1.21.5   192.168.120.100   <none>        Ubuntu 20.04.2 LTS   5.4.0-80-generic   containerd://1.5.5
worker01       Ready    storage,worker   162m   v1.21.5   192.168.120.101   <none>        Ubuntu 20.04.2 LTS   5.4.0-80-generic   containerd://1.5.5

noop - Thought you might like this

https://go-city.github.io/#/github.com/darxkies/k8s-tew

Not an issue, just wanted to share a Go source visualization I came across

Doesn't build on mac

It would be great if we could use this on a mac. As of now, to setup a remote cluster from a mac, I need to use vagrant and then pull the kubeconfig file, which means I can't use the k8s-tew commands.

velero CrashLoopBackOff unable to find minio

Commands run:

k8s-tew node-add -n node0-i 192.168.10.XX -x 0 -l controller
k8s-tew node-add -n node1 -i 192.168.10.XX -x 1 -l controller
k8s-tew node-add -n node2 -i 192.168.10.XX -x 2 -l controller
k8s-tew node-add -n node3 -i 192.168.10.XX -x 3 -l worker,storage
k8s-tew configure --public-network 192.168.10.0/24
k8s-tew generate --parallel --pull-images
k8s-tew deploy --import-images

Log output for velero pod:

time="2020-09-01T22:44:55Z" level=info msg="setting log-level to INFO" logSource="pkg/cmd/server/server.go:177"
time="2020-09-01T22:44:55Z" level=info msg="Starting Velero server v1.4.2 (56a08a4d695d893f0863f697c2f926e27d70c0c5)" logSource="pkg/cmd/server/server.go:179"
time="2020-09-01T22:44:55Z" level=info msg="No feature flags enabled" logSource="pkg/cmd/server/server.go:183"
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/crd-remap-version
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/pod
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/pv
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/service-account
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/add-pv-from-pvc
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/add-pvc-from-pod
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/change-pvc-node-selector
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/change-storage-class
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/cluster-role-bindings
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/crd-preserve-fields
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/job
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/pod
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/restic
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/role-bindings
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/service
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/service-account
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-aws kind=VolumeSnapshotter logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/aws
time="2020-09-01T22:44:55Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-aws kind=ObjectStore logSource="pkg/plugin/clientmgmt/registry.go:100" name=velero.io/aws
time="2020-09-01T22:44:55Z" level=info msg="Checking existence of namespace" logSource="pkg/cmd/server/server.go:361" namespace=backup
time="2020-09-01T22:44:55Z" level=info msg="Namespace exists" logSource="pkg/cmd/server/server.go:367" namespace=backup
time="2020-09-01T22:44:59Z" level=info msg="Checking existence of Velero custom resource definitions" logSource="pkg/cmd/server/server.go:396"
time="2020-09-01T22:45:04Z" level=info msg="All Velero custom resource definitions exist" logSource="pkg/cmd/server/server.go:430"
time="2020-09-01T22:45:04Z" level=info msg="Checking that all backup storage locations are valid" logSource="pkg/cmd/server/server.go:437"
An error occurred: some backup storage locations are invalid: backup store for location "default" is invalid: rpc error: code = Unknown desc = RequestError: send request failed
caused by: Get http://minio.backup.svc:9000/velero?delimiter=%2F&list-type=2&prefix=: dial tcp: lookup minio.backup.svc on 10.32.0.10:53: no such host

Using k8s-tew version 2.4.0-beta.5, running Ubuntu 20.04.1 LTS`, this is only effecting my worker node

Invalid cross-device link during etcd configure

Hi,

This is on Oracle Linux 7.6 but I assume this could happen on any LVM setup where /opt and /tmp are separate volumes. During the configure state, I get an invalid cross-device link error message during download of etcd.

./k8s-tew --debug generate
INFO[0000] Generated config entries
INFO[0000] Saved config filename=/etc/k8s-tew/config.yaml
INFO[0000] Copied filename=/opt/k8s-tew/bin/k8s-tew name=k8s-tew
INFO[0000] Downloading name=etcd-v3.3.12-linux-amd64.tar.gz url="https://github.com/coreos/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-amd64.tar.gz"
ERRO[0002] Generate failed error="could not rename '/tmp/etcd-v3.3.12-linux-amd64/etcd-v3.3.12-linux-amd64/etcdctl' to '/opt/k8s-tew/bin/etcd/etcdctl': rename /tmp/etcd-v3.3.12-linux-amd64/etcd-v3.3.12-linux-amd64/etcdctl /opt/k8s-tew/bin/etcd/etcdctl: invalid cross-device link"

My understanding of this error message is that a hard link cannot be made between separate partitions. I worked around this by making my K8S_TEW_BASE_DIRECTORY exist directly under the same partition (/opt/k8s-tew). I redid the configure and it completed but now everything that would be created to the the root / is not there so the systemctl commands don't work.

I could try manually merging, but wanted to report in case you had thoughts in how the hard linking is working during the etcd configure. My guess is it's like that for any resource downloaded to /tmp. Hope you can help with this. This is a really cool configuration utility and am looking forward to using it.

darxkies / k8s-tew Goto Github PK

k8s-tew's Introduction

Kubernetes - The Easier Way (k8s-tew)

TL;DR

Documentation

Caveats

Feedback

k8s-tew's People

Contributors

Stargazers

Watchers

Forkers

k8s-tew's Issues

Recommend Projects

Recommend Topics

Recommend Org