csi-driver-nfs-export's Introduction

CSI NFS Export Driver (EXPERIMENTAL)

Note: This repo is in PoC stage

Overview

This goal is to provide a "per-volume" NFS export for PersistentVolumes, both dynamically and statically.
Use an NFS-Ganesha Pod to export NFS
Use ClusterIP to mount NFS
Use CSI NFS Mounter instead of intree NFS volume

Data Flow

Highlights

Support Static Provisioning
Bind mount the local path on the data node
Implement controller.kubernetes.io/pod-deletion-cost
Delete NFS Pod and NFS SVC when NFS PVC is completely umounted
Implement NFS Clients Filter for NFS security (Ganesha only)

How to deploy

$ kubectl apply -f deploy/

$ kubectl get pod -l nfs-export.csi.k8s.io/server
NAME                                        READY   STATUS    RESTARTS   AGE
csi-nfs-export-controller-79f987457-t8rzt   3/3     Running   0          16m
csi-nfs-export-node-5h5cj                   3/3     Running   0          16m
csi-nfs-export-node-mgldw                   3/3     Running   0          16m
csi-nfs-export-node-272q7                   3/3     Running   0          16m

How to Use

Create StorageClass

By default, the default storage class with be used as DataStorageClass, here we use Rancher's local-path-provisioner as an example.

$ $ kubectl annotate sc local-path storageclass.kubernetes.io/is-default-class=true
storageclass.storage.k8s.io/local-path annotated

$ kubectl get sc local-path
NAME                   PROVISIONER              RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path (default)   rancher.io/local-path    Delete          WaitForFirstConsumer   false                  5d20h

$ kubectl apply -f example/storageclass.yaml

$ kubectl get sc nfs-export-csi
NAME             PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-export-csi   nfs-export.csi.k8s.io   Delete          Immediate           false                  38s

Create PVC

A DataPVC names nfs-<uuid> will be created together with the PVC.

$ kubectl apply -f example/pvc-dynamic.yaml

$ kubectl get pvc
NAME                                       STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
pvc-nfs-export-dynamic                     Bound     pvc-8474696b-1a51-4e9b-ab54-105f5ce2cb10   1Gi        RWX            nfs-export-csi    5s                                                                       piraeus-local-dflt-r2   12s
nfs-8474696b-1a51-4e9b-ab54-105f5ce2cb10   Pending                                                                        local-path        3s

Attach and Mount PV

An 'NFSPod' and an 'NFSService', both named nfs-<uuid>, will be created to export NFS.

$ kubectl apply -f example/deployment-dynamic.yaml

$ kubectl get pod
NAME                                         READY   STATUS    RESTARTS   AGE
nfs-6c7cb3fe-385a-4f31-9dfb-b52bfb0f1702-0   1/1     Running   0          5m16s
deployment-nginx-dynamic-7d6cf76c6f-tcrrd    1/1     Running   0          5m17s
deployment-nginx-dynamic-7d6cf76c6f-nbdbh    1/1     Running   0          5m17s
deployment-nginx-dynamic-7d6cf76c6f-hr9xj    1/1     Running   0          5m17s

$ kubectl get svc
NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                    AGE
nfs-6c7cb3fe-385a-4f31-9dfb-b52bfb0f1702   ClusterIP   10.152.183.11   <none>        2049/TCP,111/TCP,111/UDP   2m42s

csi-driver-nfs-export's People

Contributors

Stargazers

Watchers

csi-driver-nfs-export's Issues

Failed to pull image

Hi.

I try to install the csi-driver-nfs-export with a Helm chart from ed7b372 commit.

Cannot resolve daohub.ufile.ucloud.com.cn which is a CNAME pointing to 1cc8aee5.cdn.ucloud.com.cn

kubectl -n csi-driver-nfs-export describe pod/csi-nfs-export-controller-7b76f548-n2wbb
...
  Normal   Scheduled  73s               default-scheduler  Successfully assigned csi-driver-nfs-export/csi-nfs-export-controller-7b76f548-n2wbb to kube5006
  Normal   Pulling    73s               kubelet            Pulling image "k8s.m.daocloud.io/sig-storage/csi-provisioner:v3.2.0"
  Normal   Pulled     34s               kubelet            Successfully pulled image "k8s.m.daocloud.io/sig-storage/csi-provisioner:v3.2.0" in 39.014349522s
  Normal   Created    34s               kubelet            Created container provisioner
  Normal   Started    34s               kubelet            Started container provisioner
  Normal   Pulling    34s               kubelet            Pulling image "k8s.m.daocloud.io/sig-storage/livenessprobe:v2.7.0"
  Normal   Started    32s               kubelet            Started container liveness-probe
  Normal   Pulled     32s               kubelet            Successfully pulled image "k8s.m.daocloud.io/sig-storage/livenessprobe:v2.7.0" in 2.345756741s
  Normal   Created    32s               kubelet            Created container liveness-probe
  Warning  Failed     23s               kubelet            Failed to pull image "daocloud.io/piraeus/csi-nfs-export-plugin:latest": rpc error: code = Unknown desc = failed to pull and unpack image "daocloud.io/piraeus/csi-nfs-export-plugin:latest": failed to copy: httpReadSeeker: failed open: failed to do request: Get "http://daohub.ufile.ucloud.com.cn/docker/registry/v2/blobs/sha256/3f/3f75ad91a3dc0acd2d1849c0648ba1159873a9805f95d51daa89e82838037b18/data?Expires=1680517758&Signature=x7lw4ngOGAgJ07Dq5z4t1Rg0CoE%3D&UCloudPublicKey=ucloudit%40daocloud.io14149252192085992294": dial tcp: lookup daohub.ufile.ucloud.com.cn: no such host
  Normal   BackOff    22s               kubelet            Back-off pulling image "daocloud.io/piraeus/csi-nfs-export-plugin:latest"
  Warning  Failed     22s               kubelet            Error: ImagePullBackOff
  Normal   Pulling    9s (x2 over 32s)  kubelet            Pulling image "daocloud.io/piraeus/csi-nfs-export-plugin:latest"
  Warning  Failed     0s (x2 over 23s)  kubelet            Error: ErrImagePull
  Warning  Failed     0s                kubelet            Failed to pull image "daocloud.io/piraeus/csi-nfs-export-plugin:latest": rpc error: code = Unknown desc = failed to pull and unpack image "daocloud.io/piraeus/csi-nfs-export-plugin:latest": failed to copy: httpReadSeeker: failed open: failed to do request: Get "http://daohub.ufile.ucloud.com.cn/docker/registry/v2/blobs/sha256/3f/3f75ad91a3dc0acd2d1849c0648ba1159873a9805f95d51daa89e82838037b18/data?Expires=1680517780&Signature=Q6N0fnkfoLz7Cq6XHA0Ih74YuMY%3D&UCloudPublicKey=ucloudit%40daocloud.io14149252192085992294": dial tcp: lookup daohub.ufile.ucloud.com.cn: no such host

Regards.

Recommend Projects