Kopilot is a network tunnel used to proxy Kubernetes API requests to member clusters. Kopilot leverages WebSocket as the underlying connection, secured via HTTPS.
The kopilot-agent running in a member cluster will first initiate a WebSocket connection to the kopilot-hub running in the host cluster. Then, the WebSocket connection will be multiplexed with Yamux and used as the proxy channel to the Kubernetes API of the member cluster.
- Proxies Kubernetes API requests to multiple member clusters
- Only the host cluster needs to be externally addressable
- Connections are secured and encrypted via HTTPS
- Load-balances member cluster requests when multiple hub-agent connections are available
- Access to member clusters is protected by RBAC rules on the host cluster
- Runs on x86_64 or ARM64
For the host cluster:
- Kubernetes 1.16+ / minikube / kind
- cert-manager 1.0+
For member clusters:
- Kubernetes 1.16+ / minikube / kind
First, ensure that cert-manager is installed on the host cluster. If it is not installed yet, you can install it as described in the cert-manager installation documentation. Alternatively, you can simply just run the single command below:
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yamlOnce cert-manager is running, you can now deploy the kopilot-hub on the host cluster:
kubectl apply -f https://github.com/smartxworks/kopilot/releases/download/v0.3.0/kopilot.yaml
export HUB_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[0].address}')
echo $HUB_IP # change if this value is incorrect
export HUB_PORT=6443
kubectl create configmap kopilot-hub -n kopilot-system --from-literal=public_addr=$HUB_IP:$HUB_PORTFirst, create a Cluster object in the host cluster to represent one member cluster that needs to be proxied:
kubectl apply -f https://raw.githubusercontent.com/smartxworks/kopilot/master/samples/cluster.yamlThen, deploy the kopilot-agent on the member cluster:
export HUB_ADDR=$(kubectl get configmap kopilot-hub -n kopilot-system -o jsonpath='{.data.public_addr}')
export MEMBER_NAMESPACE=default
export MEMBER_NAME=sample
export MEMBER_TOKEN=$(kubectl get cluster sample -o jsonpath='{.token}')
export MEMBER_KUBECONFIG=~/.kube/member.config # change to your member cluster's kubeconfig path
curl -k "https://$HUB_ADDR/apis/subresource.kopilot.smartx.com/v1alpha1/namespaces/$MEMBER_NAMESPACE/clusters/$MEMBER_NAME/agent?token=$MEMBER_TOKEN" | kubectl apply --kubeconfig=$MEMBER_KUBECONFIG -f -Once the kopilot-agent is running, you can now send Kubernetes API requests to the member cluster from the host cluster with proper RBAC rules:
# create a kubectl pod with proper RBAC rules
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
name: kubectl
namespace: kopilot-system
---
apiVersion: v1
kind: Pod
metadata:
name: kubectl
namespace: kopilot-system
spec:
serviceAccountName: kubectl
containers:
- name: kubectl
image: bitnami/kubectl
securityContext:
runAsUser: 0
runAsGroup: 0
command:
- sleep
- infinity
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubectl
rules:
- apiGroups:
- subresource.kopilot.smartx.com
resources:
- clusters/proxy
verbs:
- "*"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubectl
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubectl
subjects:
- kind: ServiceAccount
name: kubectl
namespace: kopilot-system
EOF
# get inside the pod
kubectl exec kubectl -n kopilot-system -it -- /bin/bash
# create the member cluster's kubeconfig with service account token
export MEMBER_NAMESPACE=default
export MEMBER_NAME=sample
export MEMBER=${MEMBER_NAMESPACE}_${MEMBER_NAME}
kubectl config set-cluster $MEMBER --server=https://kubernetes.default/apis/subresource.kopilot.smartx.com/v1alpha1/namespaces/$MEMBER_NAMESPACE/clusters/$MEMBER_NAME/proxy --insecure-skip-tls-verify=true
kubectl config set-context $MEMBER --cluster=$MEMBER
kubectl config set-credentials user --token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
kubectl config set-context $MEMBER --user=user
kubectl config use-context $MEMBER
# get pods of the member cluster
kubectl get pods -AThis project is licensed under the Apache-2.0 License. See the LICENSE file for more information.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent