kubernetes-csi / csi-proxy Goto Github PK

View Code? Open in Web Editor NEW

58.0 58.0 56.0 9.73 MB

CSI Proxy utility to enable CSI Plugins on Windows

License: Apache License 2.0

Shell 20.55% Makefile 3.53% Go 74.37% Python 1.29% PowerShell 0.26%

k8s-sig-storage

csi-proxy's People

Contributors

Stargazers

Watchers

csi-proxy's Issues

Updates in the Filesystem v1beta2 API

API Changes:

Change all parameters from camelCase to underscores e.g. diskNumber -> disk_number
Make sure that the comments match what the api does
Remove the error field in the response messages
LinkPath -> CreateSymlink, IsMountPoint -> IsSymlink

Also:

change proto folder from v1 to v1beta2, check the setup in https://docs.google.com/document/d/1sBP8f_mwV0N_xRRQQGDwHZpU_nTHtpFxdi7LKSUtgX0/edit# and #120 for more details

Does not start - exits with panic

Hi,

we have compiled the csiproxy with go 1.17.6 and started it according to the instructions as a windows service.

Unfortunately it does not seem to run.
When we start it manually, it exists with a panic.

[Administrator@CHMUW-xxx-W]: C:\etc\kubernetes\logs> cat .\csi-proxy.log
Log file created at: 2022/02/16 12:14:11
Running on machine: chmuw-default-windows-zvvxd
Binary: Built with gc go1.15.14 for windows/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0216 12:14:11.704148    9872 main.go:142] Running as a Windows service.
I0216 12:14:11.721020    9872 main.go:64] Starting CSI-Proxy Server ...
I0216 12:14:11.721020    9872 main.go:65] Version: v1.0.2-36-g556e1ea
I0216 12:14:11.704660    9872 main.go:152] Windows Service initialized through SCM
I0216 12:14:11.721020    9872 main.go:84] Working directories: %v[C:\var\lib\kubelet]
Log file created at: 2022/02/16 12:29:38
Running on machine: chmuw-default-windowsxxx
Binary: Built with gc go1.17.6 for windows/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0216 12:29:38.452284   12060 main.go:142] Running as a Windows service.
I0216 12:29:38.452828   12060 main.go:152] Windows Service initialized through SCM
I0216 12:29:38.472060   12060 main.go:64] Starting CSI-Proxy Server ...
I0216 12:29:38.472060   12060 main.go:65] Version: v1.0.2-36-g556e1ea
I0216 12:29:38.472060   12060 main.go:84] Working directories: %v[C:\var\lib\kubelet]
Log file created at: 2022/02/16 12:32:32
Running on machine: chmuw-default-windows-xxx
Binary: Built with gc go1.17.6 for windows/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0216 12:32:32.055607    2064 main.go:142] Running as a Windows service.
I0216 12:32:32.055607    2064 main.go:152] Windows Service initialized through SCM
I0216 12:32:32.083475    2064 main.go:64] Starting CSI-Proxy Server ...
I0216 12:32:32.083475    2064 main.go:65] Version: v1.0.2-36-g556e1ea
I0216 12:32:32.084022    2064 main.go:84] Working directories: %v[C:\var\lib\kubelet]


[Administrator@CHMUW-DEFAULT-W]: C:\etc\kubernetes\node\bin> .\csi-proxy.exe --working-dir=c:\var\lib\kubelet
I0216 12:40:43.836545    2948 main.go:64] Starting CSI-Proxy Server ...
I0216 12:40:43.862021    2948 main.go:65] Version: v1.0.2-36-g556e1ea
I0216 12:40:43.862021    2948 main.go:84] Working directories: %v[c:\var\lib\kubelet C:\var\lib\kubelet]
panic: ([]error) 0xc000004c48

goroutine 1 [running]:
main.main()
        /root/csi-proxy/cmd/csi-proxy/main.go:73 +0x1fd


OS Name:                   Microsoft Windows Server Standard
OS Version:                10.0.19041 N/A Build 19041

Updates in the Disk v1beta3 API

API Changes:

Change all parameters from camelCase to underscores e.g. diskNumber -> disk_number
Use disk_number instead of ID, also change from string to int64
PartitionDisk: is disk partitioned formatted? (TODO: ask michelle or deep about this one)
ListDiskLocation TODO(mauricio): look for a better way to represent DiskLocation fields, it’s a map of Location to string but an empty location should be a nil pointer, check how this should be done in protobuf
ListDIskIDsResponse, change diskIDs to disk_ids
~~DiskIDs: change identifiers to map<DiskType, string>, DiskType is a new enum with only one value (page83)~~, instead change it to a message with these known types: page83 and serial_number
PartitionDisk add comment how is partitioned: “PartitionDisk initializes and partitions a disk device with the GPT partition style (if the disk has not been partitioned already) and returns the resulting volume device ID“
Rescan: (typically part of nodeStageVolume, maybe move it to sytem API)? this is just a suggestion
Change DiskStats to GetDiskStat, also change diskSize to total_bytes
Change SetAttachState to SetDiskState also change request and response
Change GetAttachState to GetDiskState, also change request and response
Make sure that the comments match what the api does (all methods)

Also:

change client API folder from v1 to v1beta3, check the setup in https://docs.google.com/document/d/1sBP8f_mwV0N_xRRQQGDwHZpU_nTHtpFxdi7LKSUtgX0/edit# and #120 for more details

community binary is missing

What happened:
The build job is broken: https://k8s-testgrid.appspot.com/sig-storage-image-build#post-csi-proxy-push-images

PartitionDisk doesn't work with GPT volumes

What happened: #128 (comment) (default to GPT instead of MBR) breaks PartitionDisk

csi-proxy/internal/server/disk/server.go

Line 79 in a0b516e

klog.V(4).Infof("Checking if disk %s is partitioned", diskID)

because PartitionDisk naively checks if any partition exists and erroneously detects the GPT 16MB reserved partition as one such partition. Result is that the UseMaximumSize partition doesn't get created and ListVolumeondisk unexpectedly returns one entry with empty string.

I0601 19:05:14.144201    5564 server.go:123] calling ListDiskIDs
I0601 19:05:16.365505    5564 server.go:122] calling PathExists with path "c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pvc-ece81bd9-b1ed-4830-98ea-bf27958ef4bd\\globalmount"
I0601 19:05:16.366181    5564 server.go:238] calling GetVolumeFromMount with request &{Mount:c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-ece81bd9-b1ed-4830-98ea-bf27958ef4bd\globalmount}
E0601 19:05:16.694188    5564 server.go:251] failed GetVolumeFromMount error getting the volume for the mount c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-ece81bd9-b1ed-4830-98ea-bf27958ef4bd\globalmount, internal error error getting volume from mount. cmd: (Get-Item -Path c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-ece81bd9-b1ed-4830-98ea-bf27958ef4bd\globalmount).Target, output: , error: <nil>
I0601 19:05:16.694464    5564 server.go:59] calling PartitionDisk with diskID "5"
I0601 19:05:18.923412    5564 server.go:69] Initializing disk 5
I0601 19:05:21.396159    5564 server.go:79] Checking if disk 5 is partitioned
I0601 19:05:23.877015    5564 server.go:93] Disk 5 already partitioned
I0601 19:05:26.353187    5564 server.go:109] calling IsVolumeFormatted with request: &{VolumeId:}
E0601 19:05:26.353187    5564 server.go:114] volume id empty

What you expected to happen: PartitionDisk works, volume gets found, formatted, and mounted, and my Pod using the volume runs.

How to reproduce it:

use latest csi-proxy with a csi driver

Anything else we need to know?:

Environment:

CSI Driver version:
Kubernetes version (use kubectl version):
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

Basic linux Makefile

At least to build binaries, and generate boilerplate code.

Image csi-driver can not pull

Hi, I execute this command:
docker pull org/csi-driver:win-v1

Then the following error reports have occurred:
Error response from daemon: pull access denied for org/csi-driver, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

But I can pull the image k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0

So how to pull the image org/csi-driver:win-v1 ?

Broken Link of `contributor cheat sheet` needs to fix

Bug Report

I have observed same kind of issue in various kubernetes-csi project.
this happens because after the localization there are too much modifications done in the various directories.
I have observed same issue in this page also.

It has one broken link of the contributes cheat sheet which needs to fix.
I will try to look in further csi repo as well and try to fix it as soon as I can

/kind bug
/assign

Use golang version from release-tools

This is using go 1.12, but release tools is currently on 1.13

Refactor the Disk API group to become a library

Add examples of Pod Security Policy(PSP) to restrict csi-proxy pipes

Csi-proxy has no mechanism to collect stats

An additional api to collect volume stats is missing to be able to implement NodeServiceCapability_RPC_GET_VOLUME_STATS

Update the integration tests to run in a HostProcess pod

Formatting and mounting a volume is much slower on windows than on linux

What happened:

CSI volume operations are much slower on windows using csi-proxy than on Linux.

What you expected to happen:
Performance of storage operations on Linux and the ones being performed by csi-proxy on Windows to be comparable. While slower performance is expected we have seen a 3x multiplier when compared to the same operations on a Linux node.

We see multiple process invocations using os.exec. Another observation is the creation of the PowerShell runspace and session for the distinct commands.

containers cannot connect to csi-proxy named pipes with containerd on Windows

Containers are failing to create when pods that mount in named pipes on Windows when running with containerd with the following error

Warning Failed 14s kubelet, 3336k8s00000000 Error: failed to generate container "fa55d3efa900b24520fa7a528e61dece7b7694dbf444a2654d846b2b3840a30f" spec: failed to generate spec: failed to resolve symlink "": CreateFile \\.\pipe\csi-proxy-filesystem-v1alpha1: All pipe instances are busy.

This works when Windows nodes are configured with docker.

Named Pipe cannot be found on windows node

What happened:
Getting the following error in my kubelet log on my windows node:

desc = "transport: Error while dialing open \\\\\\\\.\\\\\\\\\pipe\\\\\\\\\csi-proxy-filesystem-v1beta1: The system cannot find the file specified."

This results in pods requiring access to CSI storage to fail. I verified that the named pipes are indeed there on the windows node using powershell cmd '[System.IO.Directory]::GetFiles("\.\pipe\")'. The number of slashes look extremely suspicious to me. I might expect an extra pair due to log escaping, but this look excessive.

The odd thing is that I have other windows nodes where the csi-proxy is working properly. I verified that all versions of all the kubernetes containers, docker, and CSI drivers are exactly the same between the working nodes and the non-working nodes. The windows nodes are currently at the "11/2020 CU" release. They are VMs...not physical machines. I even tried "kubeadm reset" and rejoining the cluster to see if that would help. I know that vmware tools and microsoft update patches have been known to cause weird behavior. In this case, the vmware tools are not installed and the windows update is set to 11/2020 CU, which is known as a good stable CU for docker and kubernetes (since I have other windows nodes running just fine off the same version).

I also tried connecting to the pipe using a TCP --> named pipe connection tool. The connection was successful, so the pipe is definitely present. I'm wondering if the excess slashes are causing issues...but where would I dig into that being the issue?

What you expected to happen:
The pipe should be found and CSI operations can be performed.

How to reproduce it:
Kubernetes with windows nodes acting as workers.

Anything else we need to know?:

This is the same error reflected when viewing the kubernetes logs. The slashes look a little more correct.

Environment:

CSI Driver version: mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0
CSI Proxy version: 0.2.2 (also tried 0.2.1)
Kubernetes version (use kubectl version): 1.19.3
OS (e.g. from /etc/os-release): Windows Server 2019 (10.0.17763)
Docker: 1.19.11:
Kubelet: 1.19.3
Others:

Pod remains in the Pending state with error - Resize-Partition : Size Not Supported

What happened:
Created 2Gi PVC, PV is provisioned with 2Gi
Extended PVC Capacity to 3Gi

$ kubectl describe pvc example-windows-pvc-1
Name:          example-windows-pvc-1
Namespace:     default
StorageClass:  example-windows-sc
Status:        Bound
Volume:        pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: csi.vsphere.vmware.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      2Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    example-windows-pod-1
Conditions:
  Type                      Status  LastProbeTime                     LastTransitionTime                Reason  Message
  ----                      ------  -----------------                 ------------------                ------  -------
  FileSystemResizePending   True    Mon, 01 Jan 0001 00:00:00 +0000   Mon, 24 Jan 2022 11:17:06 -0800           Waiting for user to (re-)start a pod to finish file system resize of volume on node.

PV got extended to 3 Gi

$ kubectl describe pv pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67
Name:            pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67
Labels:          <none>
Annotations:     pv.kubernetes.io/provisioned-by: csi.vsphere.vmware.com
Finalizers:      [kubernetes.io/pv-protection external-attacher/csi-vsphere-vmware-com]
StorageClass:    example-windows-sc
Status:          Bound
Claim:           default/example-windows-pvc-1
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        3Gi
Node Affinity:   <none>
Message:         
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            csi.vsphere.vmware.com
    VolumeHandle:      6d5caf4c-36ef-49f1-a804-0269c785b40d
    ReadOnly:          false
    VolumeAttributes:      storage.kubernetes.io/csiProvisionerIdentity=1643013955993-8081-csi.vsphere.vmware.com
                           type=vSphere CNS Block Volume
Events:                <none>

Created Pod using this volume
Pod remains in the Pending State with error - Resize-Partition : Size Not Supported.

Name:         example-windows-pod-1
Namespace:    default
Priority:     0
Node:         tkg-vc-antrea-md-0-windows-containerd-74d9795f7b-2kg72/10.92.213.208
Start Time:   Thu, 27 Jan 2022 09:09:59 -0800
Labels:       <none>
Annotations:  <none>
Status:       Pending
IP:           
Containers:
  test-container-1:
    Container ID:  
    Image:         mcr.microsoft.com/windows/servercore:ltsc2019
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Command:
      powershell.exe
      -Command
      while (1) { Add-Content -Encoding Ascii C:\test\data.txt $(Get-Date -Format u); sleep 1 }
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /mnt/test-volume-1/ from test-volume-1 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2trft (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  test-volume-1:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  example-windows-pvc-1
    ReadOnly:   false
  kube-api-access-2trft:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              kubernetes.io/hostname=tkg-vc-antrea-md-0-windows-containerd-74d9795f7b-2kg72
Tolerations:                 node.kubernetes.io/not-ready:NoExecute for 300s
                             node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                  Age        From                                                             Message
  ----     ------                  ----       ----                                                             -------
  Normal   Scheduled               <unknown>                                                                   Successfully assigned default/example-windows-pod-1 to tkg-vc-antrea-md-0-windows-containerd-74d9795f7b-2kg72
  Normal   SuccessfulAttachVolume  41m        attachdetach-controller                                          AttachVolume.Attach succeeded for volume "pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67"
  Warning  FailedMount             40m        kubelet, tkg-vc-antrea-md-0-windows-containerd-74d9795f7b-2kg72  MountVolume.MountDevice failed while expanding volume for volume "pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67" : Expander.NodeExpand failed to expand the volume : rpc error: code = Internal desc = error when resizing filesystem on volume "6d5caf4c-36ef-49f1-a804-0269c785b40d" on node: error when resizing filesystem on devicePath \\?\Volume{3df70ab1-1cfe-4047-bd7f-a5d6e44b754e}\ and volumePath \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67\globalmount, err: rpc error: code = Unknown desc = error resizing volume. cmd: Get-Volume -UniqueId "\\?\Volume{3df70ab1-1cfe-4047-bd7f-a5d6e44b754e}\" | Get-Partition | Resize-Partition -Size 3204431360, output: Resize-Partition : Size Not Supported

error when resizing filesystem on volume "6d5caf4c-36ef-49f1-a804-0269c785b40d" on node: error when resizing filesystem on devicePath \?\Volume{3df70ab1-1cfe-4047-bd7f-a5d6e44b754e}\ and volumePath \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-021b5d9c-ba3a-4c9e-ae22-df2cbfce9e67\globalmount, err: rpc error: code = Unknown desc = error resizing volume. cmd: Get-Volume -UniqueId "\?\Volume{3df70ab1-1cfe-4047-bd7f-a5d6e44b754e}" | Get-Partition | Resize-Partition -Size 3204431360, output: Resize-Partition : Size Not Supported

What you expected to happen:

Pod should not remain in this pending state

How to reproduce it:
extend volume size on the new volume before using the volume with the pod.

Anything else we need to know?:

Environment:

CSI Driver version: vSphere CSI Driver 2.4.1
Kubernetes version (use kubectl version): v1.22.3
OS (e.g. from /etc/os-release): Windows Server 2019, Version 1809
Kernel (e.g. uname -a):
Install tools: CSI proxy v1.0.2
Others:

Run Windows node CSI Drivers as HostProcess containers

The HostProcess container feature became beta in 1.23, we'd like to leverage it in CSI Drivers which will run as privileged jobs in the Windows host, there'll be more details about the transition steps in the design doc.

By making the CSI Driver a HostProcess pod we no longer need the binary in the client/server model (although we will still support it). There are some items for the maintenance of the current client/server model of CSI Proxy.

Tasks

Beta Give feedback

Update the .github/workflows scripts to work with both v1.x and master #222
Update the Prow jobs to work with the v1.x branch and the master branch #223
Update the integration scripts to work with the v1.x and master branches #224
Release CSI Proxy v1.1.2 #225
Design doc: Using Windows HostProcess containers in CSI Drivers running in Windows nodes @mauriciopoppe created kubernetes/enhancements#3636
Blogpost draft: https://docs.google.com/document/d/18WwuKSONG_qkD3vaoppL_ikeDL0Ms0Wwv7HsyN8Oel0/edit
Options

Items for the refactor of CSI Proxy to become a go library:

Tasks

Beta Give feedback

Items to use the new CSI Proxy library in a CSI Driver (PD CSI Driver):

Tasks

Beta Give feedback

Update the Windows node manifests adding the HostProcess Pod fields kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1061

lifecycle/frozen
Refactor the Windows node code to use the CSI Proxy library pkg/api code instead of the client/ code kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1062

lifecycle/frozen
Verify that the E2E jobs are passing with the Windows node component running as a HostProcess Pod kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1063

lifecycle/frozen
Migrate CSI Proxy to v2 kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1070
Try https://github.com/microsoft/windows-host-process-containers-base-image as a new base image after the POC is completed
Options

/assign @mauriciopoppe

Use common infra from release-tools/build.make

Implement resize APIs for Disk and Volume groups

Implement an API to resize disks and volume objects.

Updates in the SMB v1beta2 API

API Changes:

Change all parameters from camelCase to underscores e.g. diskNumber -> disk_number
Remove string error=1 from all the messages
Make sure that the comments match what the api does

Also:

change proto folder from v1 to v1beta2, check the setup in https://docs.google.com/document/d/1sBP8f_mwV0N_xRRQQGDwHZpU_nTHtpFxdi7LKSUtgX0/edit# and #120 for more details

Updates in the Volume v1beta3 API

API Changes

Change all parameters from camelCase to underscores e.g. diskNumber -> disk_number
Make sure that the comments match what the api does
ListVolumesOnDisk to support multiple partition case, need to add parameters related to partition e.g. send another parameter called partition_number
Use disk number instead of ID, also change from string to int64
Change DismountVolume to UnmountVolume (TODO: confirm this change)
Change path to target_path in MountVolume and UnmountVolume
isVolumeFormatted: Update comment to “checks if a volume is formatted.”
FormatVolume: update comment to “formats a volume with NTFS.”
ResizeVolume: change size to size_bytes
VolumeStats: GetVolumeStats, also change the request and response to GetVolumeStatsRequest and GetVolumeStatsResponse
VolumeStatsResponse: volumeSize to total_bytes and volumeUsedSize to used_bytes https://github.com/container-storage-interface/spec/blob/da58351ba3d7baf850877be3425e76ff30645d33/csi.proto#L1410-L1430
Change GetVolumeDiskNumber to GetDiskNumberFromVolumeID, also change GetDiskNumberFromVolumeIDRequest and GetDiskNumberFromVolumeIDResponse
VolumeIDFromMountRequest to VolumeIDFromTargetPathRequest Change, change the parameter to target_path
WriteVolumeCache -- update comment to “Writes the file system cache to disk with the given volume id”

Also:

change proto folder from v1 to v1beta3, check the setup in docs.google.com/document/d/1sBP8f_mwV0N_xRRQQGDwHZpU_nTHtpFxdi7LKSUtgX0/edit# and #120 for more details

Update the integration scripts to work with the v1.x and master branches

Update the scripts/run-integration.sh script to work in both branches, I think we might need a different version for HostProcess containers.

does csi-proxy support domain name resolve?

kubernetes-csi/csi-driver-smb#160

windows: 2019
k8s: v1.17.2

when i use
source: \sfs-nas1.cn-south-1c.xxx.com\share-2d58d4ca
is does not work, but when i use
source: \10.12.13.5\share-2d58d4ca
it works

thanks for your help!

release-tools subtree is broken

Because this repo overrode the Makefile to not use the release-tools/build.make rules, it never ran any of the verify-* scripts during pull jobs, such as verify-subtree.sh.

This let #22 merge, which directly modified release-tools/, which we don't want.

@pohly, I remember we had this problem in another repo and we relaxed the verify script to allow reverts to work. Can we do that here? Revert #22, and then update release-tools + enable the verify scripts?

more docs

Placeholder for a second doc iteration, adding new provider info and known implementations (vsphere), improving asciiflow diagram stolen from @ddebroy , and so on....

either me or @gab-satchi can do this

CSI-Proxy fails to return the disks through disk.ListDiskIDs and disk.ListDiskLocations in a VM with only one disk

What happened:

In a VM with only a single disk (like when it's a new VM) the integration tests fail while trying to get the list of disks, this command:

(Get-Disk | Select Path, SerialNumber) | ConvertTo-Json

Will return an object if there's only one disk but an array of objects if there are many disks, the go code is always unmarshalling an array so if it only has one disk it fails during unmarshalling (silently), the result is that this method returns no disks when there's one disk.

csi-proxy/internal/os/disk/api.go

Lines 286 to 305 in e7c1df0

 func (imp DiskAPI) ListDiskIDs() (map[uint32]shared.DiskIDs, error) { 

 // sample response 

 // [ 

 // { 

 // "Path": "\\\\?\\scsi#disk\u0026ven_google\u0026prod_persistentdisk#4\u002621cb0360\u00260\u0026000100#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}", 

 // "SerialNumber": " " 

 // }, 

 // { 

 // "Path": "\\\\?\\scsi#disk\u0026ven_msft\u0026prod_virtual_disk#2\u00261f4adffe\u00260\u0026000001#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}", 

 // "SerialNumber": null 

 // }, 

 cmd := "(Get-Disk | Select Path, SerialNumber) | ConvertTo-Json" 

 out, err := runExec(cmd) 

 if err != nil { 

 return nil, fmt.Errorf("Could not query disk paths") 

 } 

 outString := string(out) 

 disks := []Disk{} 

 json.Unmarshal([]byte(outString), &disks)

Document the go structs used by clients when they import CSI Proxy

Document files like https://github.com/kubernetes-csi/csi-proxy/blob/master/pkg/{api-group}/types.go e.g. https://github.com/kubernetes-csi/csi-proxy/blob/master/pkg/filesystem/types.go

Volume and csi-proxy-api-gen E2E tests are skipped in github actions

What happened:
Some E2E tests are skipped if the test detects that it's running inside a Github Action workflow

csi-proxy/integrationtests/volume_test.go

Line 373 in c249530

skipTestOnCondition(t, isRunningOnGhActions())
csi-proxy/integrationtests/csi_api_gen_test.go

Line 21 in c249530

skipTestOnCondition(t, isRunningOnGhActions())

What you expected to happen:
E2E tests shouldn't be skipped, it makes it hard to iterate if you don't know what you broke

How to reproduce it:
Comment the line

csi-proxy/integrationtests/volume_test.go

Line 373 in c249530

skipTestOnCondition(t, isRunningOnGhActions())

and send a PR that triggers the presubmit job, there's a powershell command that fails because Hyper-V is disabled.

See https://github.com/kubernetes-csi/csi-proxy/runs/2605742614?check_suite_focus=true for an example

Anything else we need to know?:
There are some commands that can't run in the Github Actions VM because it has hyper-V disabled, this feature was disabled in actions/runner-images#2525 and it looks like it's not coming back.

Remove dependency on Diskutil.exe used by GetDiskNumberByName API

Currently, PD disk id is not available through powershell command. As a workaround we use a C code disktutil.exe to get Page 83 ID based mapping. We want to have a Go version of the C code of diskutil.exe using go-bindings (https://medium.com/jettech/breaking-all-the-rules-using-go-to-call-windows-api-2cbfd8c79724)

Possible to reformat a disk that has a Linux fs on it

What happened:
The implementation of IsVolumeFormatted only checks for ntfs filesystem, and it returns false if it doesn't find ntfs. That means it returns unformatted if there's a linux filesystem on it, and then subsequently calling FormatVolume can override the filesystem.

More discussion in https://app.slack.com/client/T09NY5SBT/CN5JCCW31/thread/C8EJ01Z46-1603793388.212200

What you expected to happen:
Format should error if a filesystem is already on the disk.

panic: protobuf tag not enough fields in PathExistsResponse.state

What happened:
related smb code: https://github.com/kubernetes-csi/csi-driver-smb/blob/8ec0d721cf63e93a324689c17868fd433d193afd/pkg/mounter/safe_mounter_windows.go#L221-L224

PS C:\Users\xiazhang\Downloads\csi-proxy-v1.0.0-rc.1.tar\bin> .\csi-proxy.exe --kubelet-path C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-driver-smb\pkg\smb\ --v 10
I0705 09:29:22.634987   26268 main.go:54] Starting CSI-Proxy Server ...
I0705 09:29:22.636127   26268 main.go:55] Version: v1.0.0-rc.1-0-g327ebfb
I0705 09:29:31.082178   26268 server.go:105] Request: PathExists with path="C:\\Users\\xiazhang\\go\\src\\github.com\\kubernetes-csi\\csi-driver-smb\\pkg\\smb\\test-csi"

C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-driver-smb\pkg\smb>go test -run TestCreateVolume
panic: protobuf tag not enough fields in PathExistsResponse.state:  [recovered]
        panic: protobuf tag not enough fields in PathExistsResponse.state:

goroutine 14 [running]:
testing.tRunner.func1(0xc00022c600)
        c:/go/src/testing/testing.go:874 +0x3aa
panic(0x155dcc0, 0xc0004170b0)
        c:/go/src/runtime/panic.go:679 +0x1c0
github.com/golang/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc0003c0140)
        C:/Users/xiazhang/go/pkg/mod/github.com/golang/[email protected]/proto/table_unmarshal.go:332 +0x16aa
github.com/golang/protobuf/proto.(*unmarshalInfo).unmarshal(0xc0003c0140, 0xc000332090, 0xc000366088, 0x2, 0x2, 0x40d6bf, 0x20)
        C:/Users/xiazhang/go/pkg/mod/github.com/golang/[email protected]/proto/table_unmarshal.go:136 +0xde7
github.com/golang/protobuf/proto.(*InternalMessageInfo).Unmarshal(0xc000364120, 0x4a14130, 0xc000332090, 0xc000366088, 0x2, 0x2, 0x3, 0x0)
        C:/Users/xiazhang/go/pkg/mod/github.com/golang/[email protected]/proto/table_unmarshal.go:63 +0x6d
github.com/golang/protobuf/proto.(*Buffer).Unmarshal(0xc000368458, 0x4a14130, 0xc000332090, 0x0, 0x0)
        C:/Users/xiazhang/go/pkg/mod/github.com/golang/[email protected]/proto/decode.go:424 +0x1f3
google.golang.org/grpc/encoding/proto.codec.Unmarshal(0xc000366088, 0x2, 0x2, 0x16acb80, 0xc000332090, 0xc00034f040, 0x5159a3)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:93 +0x135
google.golang.org/grpc.recv(0xc0002ef000, 0x28ec538, 0x26670b0, 0xc00022c700, 0x0, 0x0, 0x16acb80, 0xc000332090, 0x400000, 0x0, ...)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/rpc_util.go:711 +0x116
google.golang.org/grpc.(*csAttempt).recvMsg(0xc0002e0980, 0x16acb80, 0xc000332090, 0x0, 0xc000043e00, 0x54)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/stream.go:885 +0xf4
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0002e0980, 0x54, 0x54)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/stream.go:736 +0x4d
google.golang.org/grpc.(*clientStream).withRetry(0xc0002d9320, 0xc00034f310, 0xc00034f2e0, 0xc000043e00, 0xc000185e00)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/stream.go:594 +0xa3
google.golang.org/grpc.(*clientStream).RecvMsg(0xc0002d9320, 0x16acb80, 0xc000332090, 0x0, 0x0)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/stream.go:735 +0x10a
google.golang.org/grpc.invoke(0x1a4e740, 0xc00003a0e8, 0x183aedc, 0x19, 0x16acac0, 0xc000304480, 0x16acb80, 0xc000332090, 0xc000222a80, 0x0, ...)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/call.go:73 +0x142
google.golang.org/grpc.(*ClientConn).Invoke(0xc000222a80, 0x1a4e740, 0xc00003a0e8, 0x183aedc, 0x19, 0x16acac0, 0xc000304480, 0x16acb80, 0xc000332090, 0x0, ...)
        C:/Users/xiazhang/go/pkg/mod/google.golang.org/[email protected]/call.go:37 +0x1ba
github.com/kubernetes-csi/csi-proxy/client/api/filesystem/v1.(*filesystemClient).PathExists(0xc0002f5d20, 0x1a4e740, 0xc00003a0e8, 0xc000304480, 0x0, 0x0, 0x0, 0x8898ea, 0x2646980, 0x1c0008)
        C:/Users/xiazhang/go/pkg/mod/github.com/kubernetes-csi/csi-proxy/[email protected]/api/filesystem/v1/api.pb.go:840 +0xdb
github.com/kubernetes-csi/csi-proxy/client/groups/filesystem/v1.(*Client).PathExists(...)
        C:/Users/xiazhang/go/pkg/mod/github.com/kubernetes-csi/csi-proxy/[email protected]/groups/filesystem/v1/client_generated.go:81
github.com/kubernetes-csi/csi-driver-smb/pkg/mounter.(*CSIProxyMounter).ExistsPath(0xc0002f5e20, 0xc000043b60, 0x52, 0x0, 0x0, 0x0)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/mounter/safe_mounter_windows.go:221 +0x198
github.com/kubernetes-csi/csi-driver-smb/pkg/mounter.(*CSIProxyMounter).IsLikelyNotMountPoint(0xc0002f5e20, 0xc000043b60, 0x52, 0xc00028fa40, 0x50, 0x50)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/mounter/safe_mounter_windows.go:164 +0xff
github.com/kubernetes-csi/csi-driver-smb/pkg/smb.(*Driver).ensureMountPoint(0xc0002c0c80, 0xc000043b60, 0x52, 0xc0002eeea0, 0x0, 0x0)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/smb/nodeserver.go:325 +0x6d
github.com/kubernetes-csi/csi-driver-smb/pkg/smb.(*Driver).NodeStageVolume(0xc0002c0c80, 0x1a4e740, 0xc00003a100, 0xc000327bf8, 0x0, 0x0, 0x0)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/smb/nodeserver.go:183 +0x7b9
github.com/kubernetes-csi/csi-driver-smb/pkg/smb.(*Driver).internalMount(0xc0002c0c80, 0x1a4e740, 0xc00003a100, 0xc000304440, 0xc000304380, 0xc0002d1f20, 0x0, 0x0)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/smb/controllerserver.go:243 +0x319
github.com/kubernetes-csi/csi-driver-smb/pkg/smb.(*Driver).CreateVolume(0xc0002c0c80, 0x1a4e740, 0xc00003a100, 0xc0002f72d0, 0x0, 0x0, 0x0)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/smb/controllerserver.go:95 +0x321
github.com/kubernetes-csi/csi-driver-smb/pkg/smb.TestCreateVolume.func1(0xc00022c600)
        C:/Users/xiazhang/go/src/github.com/kubernetes-csi/csi-driver-smb/pkg/smb/controllerserver_test.go:165 +0x110
testing.tRunner(0xc00022c600, 0xc00028f9f0)
        c:/go/src/testing/testing.go:909 +0xd0
created by testing.(*T).Run
        c:/go/src/testing/testing.go:960 +0x357
exit status 2
FAIL    github.com/kubernetes-csi/csi-driver-smb/pkg/smb        0.327s

@mauriciopoppe
cc @jingxu97

What you expected to happen:

How to reproduce it:

run csi-proxy on Windows node

download https://github.com/andyzhangx/demo/raw/master/windows/csi-proxy-v1.0.0-rc.1.tar.gz

.\csi-proxy.exe --kubelet-path C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-driver-smb\pkg\smb\ --v 10

run TestCreateVolume which invokes csi-proxy PathExists

git clone https://github.com/kubernetes-csi/csi-driver-smb.git
cd csi-driver-smb
git fetch origin pull/319/head:319
git checkout 319
cd pkg\smb
C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-driver-smb\pkg\smb>go test -run TestCreateVolume

Anything else we need to know?:

Environment:

CSI Driver version: v1.0.0-rc.1
Kubernetes version (use kubectl version): n/a
OS (e.g. from /etc/os-release): Windows 10
Kernel (e.g. uname -a):
Install tools:
Others:

CSIProxy isn't igniting SMBGlobalMapping command

What happened:

CSIProxy isn't igniting SMBGLobalMapping command

As the mount isn't happening on the host, it throws a timeout; this sounds to be the root cause. CSIProxy isn't igniting the command. However, I can successfully run the SMB-GlobalMapping command manually.

Also, CSIProxy isn't detecting if an existing SMBGlobalMapping is already mounted. I did the test having the SMBGlobalMapping manually mounted.

CSIProxy Log:

Log file created at: 2022/04/08 13:16:45
Running on machine: EC2AMAZ-5C7TE01
Binary: Built with gc go1.18 for windows/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0408 13:16:45.457061    2776 main.go:142] Running as a Windows service.
I0408 13:16:45.457061    2776 main.go:152] Windows Service initialized through SCM
I0408 13:16:45.516779    2776 main.go:64] Starting CSI-Proxy Server ...
I0408 13:16:45.516779    2776 main.go:65] Version: 
I0408 13:16:45.519451    2776 main.go:84] Working directories: %v[C:\var\lib\kubelet]
I0408 13:18:10.381196    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:18:10.381196    2776 server.go:231] Request: IsSymlink with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:18:10.381739    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:18:10.384371    2776 server.go:153] Request: Rmdir with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:18:10.384901    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb"
I0408 13:18:10.387008    2776 server.go:36] calling NewSmbGlobalMapping with remote path "\\\\fs-002d771bfd375aac6.marciomorales.local\\share\\windowscontainer"
I0408 13:18:11.415292    2776 server.go:69] Remote \\fs-002d771bfd375aac6.marciomorales.local\share\windowscontainer not mapped. Mapping now!
I0408 13:25:33.641010    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:25:33.641544    2776 server.go:231] Request: IsSymlink with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:25:33.642078    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:25:33.642078    2776 server.go:153] Request: Rmdir with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb\\globalmount"
I0408 13:25:33.642609    2776 server.go:114] Request: PathExists with path="c:\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\pv-smb"
I0408 13:25:33.642609    2776 server.go:36] calling NewSmbGlobalMapping with remote path "\\\\fs-002d771bfd375aac6.marciomorales.local\\share\\"
I0408 13:25:34.404591    2776 server.go:69] Remote \\fs-002d771bfd375aac6.marciomorales.local\share\ not mapped. Mapping now!

kubelet log:

Events:
  Type     Reason             Age              From                     Message
  ----     ------             ----             ----                     -------
  Normal   Scheduled          2m8s             default-scheduler        Successfully assigned default/busybox-smb-76db684959-qlksd to ip-172-31-6-224.ec2.internal
  Normal   ResourceAllocated  2m8s             vpc-resource-controller  Allocated Resource vpc.amazonaws.com/PrivateIPv4Address: 172.31.27.96/19 to the pod
  Warning  FailedMount        8s               kubelet                  MountVolume.MountDevice failed for volume "pv-smb" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
  Warning  FailedMount        5s               kubelet                  Unable to attach or mount volumes: unmounted volumes=[smb], unattached volumes=[smb kube-api-access-pzq67]: timed out waiting for the condition
  Warning  FailedMount        0s (x4 over 8s)  kubelet                  MountVolume.MountDevice failed for volume "pv-smb" : rpc error: code = Aborted desc = An operation with the given Volume ID eks-test-08 already exists

What you expected to happen:

CSIProxy to ignite SMBGlobalMapping command

How to reproduce it:

Using CSI-SMB-Driver.

Anything else we need to know?:

Environment:

CSI Driver version: 1.1.1
Kubernetes version (use kubectl version): 1.22
OS (e.g. from /etc/os-release): Windows Server 2019
Kernel (e.g. uname -a): 10.0.17763 N/A Build 17763
Install tools:
Others: containerd runtime

Write generator + ps1 script to run it

See #7 (comment)

Daemonset deployment failing - ImagePullBackOff

What happened: Container images not available on the repository.

What you expected to happen:
Successfully pull the following images:

org/csi-driver:win-v1
gke.gcr.io/csi-node-driver-registrar:win-v1

How to reproduce it:

Just run the Daemonset deployment YAML file on the readme.

Anything else we need to know?:

The deployment YAML file uses a ServiceAccount called csi-node-sa which needs to be created before running the Daemontset deployment, otherwise it will fail. There is no instruction saying that needs to be created.

Environment:

CSI Driver version:
Kubernetes version (use kubectl version): 1.20
OS (e.g. from /etc/os-release): WIndows Server 2019 LTSC
Kernel (e.g. uname -a):
Install tools:
Others:

Kubectl describe pod attached (log.txt)
Log.txt

Refactor the FS API group to become a library

[feature request] implement GetVolumeStats for smb client

Is your feature request related to a problem?/Why is this needed

Describe the solution you'd like in detail

We could find there is GetVolumeStats for volume client which is mainly for getting disk usage, it does not work for smb mount:

csi-proxy/pkg/os/volume/api.go

Lines 201 to 225 in 48ba409

 // GetVolumeStats - retrieves the volume stats for a given volume 

 func (VolumeAPI) GetVolumeStats(volumeID string) (int64, int64, error) { 

 // get the size and sizeRemaining for the volume 

 cmd := fmt.Sprintf("(Get-Volume -UniqueId \"%s\" | Select SizeRemaining,Size) | ConvertTo-Json", volumeID) 

 out, err := utils.RunPowershellCmd(cmd) 

 if err != nil { 

 return -1, -1, fmt.Errorf("error getting capacity and used size of volume. cmd: %s, output: %s, error: %v", cmd, string(out), err) 

 } 

 var getVolume map[string]int64 

 outString := string(out) 

 err = json.Unmarshal([]byte(outString), &getVolume) 

 if err != nil { 

 return -1, -1, fmt.Errorf("out %v outstring %v err %v", out, outString, err) 

 } 

 var volumeSizeRemaining int64 

 var volumeSize int64 

 volumeSize = getVolume["Size"] 

 volumeSizeRemaining = getVolume["SizeRemaining"] 

 volumeUsedSize := volumeSize - volumeSizeRemaining 

 return volumeSize, volumeUsedSize, nil 

 }

PS C:\Users\azureuser> Get-Volume

DriveLetter FriendlyName      FileSystemType DriveType HealthStatus OperationalStatus SizeRemaining      Size
----------- ------------      -------------- --------- ------------ ----------------- -------------      ----
                              NTFS           Fixed     Healthy      OK                     19.82 GB  19.87 GB
                              NTFS           Fixed     Healthy      OK                     19.84 GB  19.87 GB
                              NTFS           Fixed     Healthy      OK                     19.82 GB  19.87 GB
            System Reserved   NTFS           Fixed     Healthy      OK                    466.45 MB    500 MB
C           Windows           NTFS           Fixed     Healthy      OK                     108.4 GB 127.51 GB
D           Temporary Storage NTFS           Fixed     Healthy      OK                     24.02 GB     32 GB
E                             Unknown        CD-ROM    Healthy      Unknown                     0 B       0 B
                              NTFS           Fixed     Healthy      OK                     19.82 GB  19.87 GB

for smb mount, it's using New-SmbGlobalMapping, not sure how to get usage on Windows

PS C:\Users\azureuser> Get-SmbGlobalMapping -RemotePath \\xxx.file.core.windows.net\pvc-0ce04732-33d4-4e02-8df0-b3fd06bbde8d

Status Local Path Remote Path
------ ---------- -----------
OK                \\xxx.file.core.windows.net\pvc-0ce04732-33d4-4e02-8df0-b3fd06bbde8d

Describe alternatives you've considered

Additional context

Release CSI Proxy v1.1.2

The release should happen in master, we'll later add supporting tooling if needed for v1.x

Add request completion logs in the server layer

Most of the server request handlers aren't logging when they're done fulfilling a request, this leads to issues understanding if a request is completed or stuck in a request like #206. We could add a line like klog.V(2).Infof("calling NewSmbGlobalMapping with remote path %q", request.RemotePath) at the end.

Unable to determine the size of a volume after resize

When calling resize the NodeExpandVolumeResponse requires the capacitybytes after resize. Currently there is no way to retrieve this information. Perhaps adding the resulting capacity in

message ResizeVolumeResponse {
// Intentionally empty
}

Password field should be hidden in the SMB NewSmbGlobalMappingRequest Proto

This field should be hidden

csi-proxy/client/api/smb/v1beta2/api.proto

Line 34 in c249530

string password = 4;

Investigate how to update csi-proxy's build process to include additional proto plugins like in https://github.com/container-storage-interface/spec/blob/da58351ba3d7baf850877be3425e76ff30645d33/lib/go/Makefile#L134-L144
Extend the FieldOptions field like https://github.com/container-storage-interface/spec/blob/da58351ba3d7baf850877be3425e76ff30645d33/csi.proto#L23-L32
Use the field option in the password field

Invalid version error on `go get -v github.com/kubernetes-csi/csi-proxy`

What happened:
I am currently experimenting with CSI-Proxy, so in my package, I tried to get csi-proxy using:

go get -v github.com/kubernetes-csi/csi-proxy

I am using windows 10 and go version is: go version go1.15.2 windows/amd64
I get this error:

$ go get github.com/kubernetes-csi/csi-proxy
go: github.com/kubernetes-csi/csi-proxy upgrade => v0.2.1
go get: github.com/kubernetes-csi/[email protected] requires
        github.com/kubernetes-csi/csi-proxy/[email protected]: invalid version: unknown revision 000000000000

What you expected to happen:
I am expecting the go get to complete.

How to reproduce it:
On windows machine with go version 1.15, do: go get -v github.com/kubernetes-csi/csi-proxy
Anything else we need to know?:

Environment:

CSI Driver version: latest master
Kubernetes version (use kubectl version): -
OS (e.g. from /etc/os-release): Windows
Kernel (e.g. uname -a): 10
Install tools:
Others: go version: 1.15

CSI-Proxy crashes with error: panic: The service process could not connect to the service controller when -windows-service argument is passed

What happened:
I tried to run CSI-Proxy with the argument -windows-service but then it crashes with this log.

panic: The service process could not connect to the service controller.

goroutine 1 [running]:
main.main()
        C:/Users/manoh/github/csi-proxy/cmd/csi-proxy/main.go:41 +0x27c

What you expected to happen:
I am expecting the binary to run as windows service.

How to reproduce it:
the csi-proxy binary: https://kubernetesartifacts.azureedge.net/csi-proxy/v0.2.1/binaries/csi-proxy.tar.gz with the argument -windows-service

Environment:

CSI Driver version: v0.2.1
Kubernetes version (use kubectl version):
OS (e.g. from /etc/os-release): windows 10
Kernel (e.g. uname -a):
Install tools:
Others:

cc: @andyzhangx

Setup CI jobs

Setup prow job that builds and runs unit tests, go fmt, vendor, etc.

Unsure if we still need travis CI.

/assign

Add a way to query BIOS DMI in csi-proxy

What happened:
To support CSI plugins like vsphere, csi-proxy needs to be able to surface BIOS DMI details to the plugin. Please enable this support, potentially through a new system API group. This should do something identical to: https://github.com/kubernetes/kubernetes/blob/103e926604de6f79161b78af3e792d0ed282bc06/staging/src/k8s.io/legacy-cloud-providers/vsphere/vsphere_util_windows.go#L29

What you expected to happen:
There is no way to surface BIOS DMI details today.

How to reproduce it:

Anything else we need to know?:

Environment:

CSI Driver version: NA
Kubernetes version (use kubectl version): NA
OS (e.g. from /etc/os-release): Windows
Kernel (e.g. uname -a): NA
Install tools:
Others:

Allow running only certain APIs

For the sake of being able to run several versions of csi-proxy.exe alongside each other.

See #7 (comment)

High cpu usage of powershell processes triggered by csi-proxy

What happened:
We are doing some performance tests for vsphere-csi-driver (https://github.com/kubernetes-sigs/vsphere-csi-driver) on k8s cluster with 20 windows worker nodes. Each node has 4 CPUs and 16GB of memory. k8s version is 1.22.3, windows version is Windows Server 2019. the vsphere csi node plugin uses csi-proxy for privileged operations on host devices. The test repeatedly issues CreatePod followed by DeletePod. Each Pod just starts one container with image "mcr.microsoft.com/oss/kubernetes/pause:1.4.1", and mounts one persistent volume.

During the test we observed high cpu usage on the windows worker nodes, and most of the cpu usage came from the powershell processes triggered by csi-proxy. With 1.4 Pod creations per second, the average cpu usage on each node is about 75% of the total cpu capacity, the CPU cost of powershell processes triggered by csi-proxy is about 45% of the total CPU capacity; With 2.4 Pod creations per second, the cpu usage on each node reached almost 100%, and the CPU cost of powershell processes triggered by csi-proxy is about 68%. As a comparison, for vsphere-csi-driver on Linux nodes, with 3 Pod creations per second the cpu usage on each node is constantly below 10%.

What you expected to happen:
Reduce the cpu cost of csi-proxy.

How to reproduce it:
deploy vsphere-csi-driver on k8s cluster with windows nodes, and create Pod with PV mount, then delete Pod.

Anything else we need to know?:

Environment:

CSI Driver version: vsphere CSI 2.5
Kubernetes version (use kubectl version): 1.22.3
OS (e.g. from /etc/os-release): Windows Server 2019
Kernel (e.g. uname -a):
Install tools:
Others:

Update the Prow jobs to work with the v1.x branch and the master branch

Same as #222 but updating:

https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-csi/csi-proxy/csi-proxy-manual-config.yaml

fix unit test failure on Windows

C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-proxy>go test -v -race ./internal/...
?       github.com/kubernetes-csi/csi-proxy/internal/os/disk    [no test files]
=== RUN   TestPathValid
--- PASS: TestPathValid (0.51s)
PASS
ok      github.com/kubernetes-csi/csi-proxy/internal/os/filesystem      1.604s
?       github.com/kubernetes-csi/csi-proxy/internal/os/smb     [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/os/volume  [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server     [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/disk        [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/disk/internal       [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/disk/internal/v1alpha1      [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/disk/internal/v1beta1       [no test files]
# github.com/kubernetes-csi/csi-proxy/internal/server/volume [github.com/kubernetes-csi/csi-proxy/internal/server/volume.test]
internal\server\volume\server_test.go:51:2: not enough arguments to return
        have (number, number, nil)
        want (int64, int64, int64, error)
internal\server\volume\server_test.go:99:29: cannot use volAPI (type *fakeVolumeAPI) as type API in argument to NewServer:
        *fakeVolumeAPI does not implement API (missing GetVolumeDiskNumber method)
=== RUN   TestMkdirWindows
E0907 09:50:11.894433    5124 server.go:153] failed validatePathWindows path: C:\foo\bar is not within context path: C:\var\lib\kubelet\pods
E0907 09:50:11.896433    5124 server.go:153] failed validatePathWindows path: C:\foo\bar is not within context path: C:\var\lib\kubelet\plugins
E0907 09:50:11.897433    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin\pv1:foo
E0907 09:50:11.897433    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1/foo
E0907 09:50:11.897433    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin\pv1*foo
E0907 09:50:11.897433    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1?foo
E0907 09:50:11.898434    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin|pv1\foo
E0907 09:50:11.898434    5124 server.go:153] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1\..\..\..\system32
E0907 09:50:11.898434    5124 server.go:153] failed validatePathWindows invalid character \ at beginning of path: \\csi-plugin\..\..\..\system32
E0907 09:50:11.898434    5124 server.go:153] failed validatePathWindows not an absolute Windows path: pv1\foo
--- PASS: TestMkdirWindows (0.01s)
    server_test.go:135: test case: path outside of pod context with pod context set
    server_test.go:135: test case: path inside pod context with pod context set
    server_test.go:135: test case: path outside of plugin context with plugin context set
    server_test.go:135: test case: path inside plugin context with plugin context set
    server_test.go:135: test case: path with invalid character `:` beyond drive letter prefix
    server_test.go:135: test case: path with invalid character `/`
    server_test.go:135: test case: path with invalid character `*`
    server_test.go:135: test case: path with invalid character `?`
    server_test.go:135: test case: path with invalid character `|`
    server_test.go:135: test case: path with invalid characters `..`
    server_test.go:135: test case: path with invalid prefix `\`
    server_test.go:135: test case: relative path
=== RUN   TestRmdirWindows
E0907 09:50:11.899433    5124 server.go:175] failed validatePathWindows path: C:\foo\bar is not within context path: C:\var\lib\kubelet\pods
E0907 09:50:11.899433    5124 server.go:175] failed validatePathWindows path: C:\foo\bar is not within context path: C:\var\lib\kubelet\plugins
E0907 09:50:11.900432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin\pv1:foo
E0907 09:50:11.900432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1/foo
E0907 09:50:11.900432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin\pv1*foo
E0907 09:50:11.900432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1?foo
E0907 09:50:11.900432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\plugins\csi-plugin|pv1\foo
E0907 09:50:11.901432    5124 server.go:175] failed validatePathWindows path contains invalid characters: C:\var\lib\kubelet\pods\pv1\..\..\..\system32
E0907 09:50:11.901432    5124 server.go:175] failed validatePathWindows invalid character \ at beginning of path: \\csi-plugin\..\..\..\system32
E0907 09:50:11.901432    5124 server.go:175] failed validatePathWindows not an absolute Windows path: pv1\foo
--- PASS: TestRmdirWindows (0.00s)
    server_test.go:253: test case: path outside of pod context with pod context set
    server_test.go:253: test case: path inside pod context with pod context set
    server_test.go:253: test case: path outside of plugin context with plugin context set
    server_test.go:253: test case: path inside plugin context with plugin context set
    server_test.go:253: test case: path with invalid character `:` beyond drive letter prefix
    server_test.go:253: test case: path with invalid character `/`
    server_test.go:253: test case: path with invalid character `*`
    server_test.go:253: test case: path with invalid character `?`
    server_test.go:253: test case: path with invalid character `|`
    server_test.go:253: test case: path with invalid characters `..`
    server_test.go:253: test case: path with invalid prefix `\`
    server_test.go:253: test case: relative path
PASS
ok      github.com/kubernetes-csi/csi-proxy/internal/server/filesystem  2.545s
?       github.com/kubernetes-csi/csi-proxy/internal/server/filesystem/internal [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/filesystem/internal/v1alpha1        [no test files]
=== RUN   TestNewSmbGlobalMapping
E0907 09:50:11.893431   23044 server.go:39] remote path is empty
--- PASS: TestNewSmbGlobalMapping (0.00s)
PASS
ok      github.com/kubernetes-csi/csi-proxy/internal/server/smb 2.436s
?       github.com/kubernetes-csi/csi-proxy/internal/server/smb/internal        [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/smb/internal/v1alpha1       [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/types       [no test files]
FAIL    github.com/kubernetes-csi/csi-proxy/internal/server/volume [build failed]
?       github.com/kubernetes-csi/csi-proxy/internal/server/volume/internal     [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/volume/internal/v1alpha1    [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/server/volume/internal/v1beta1     [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/shared/disk        [no test files]
?       github.com/kubernetes-csi/csi-proxy/internal/utils      [no test files]
FAIL

Update the .github/workflows scripts to work with both v1.x and master

Maintenance of CSI Proxy v1 is going to happen in the v1.x branch, we need to have variations of the windows.yml script that run in v1.x and in master

	func (imp DiskAPI) ListDiskIDs() (map[uint32]shared.DiskIDs, error) {
	// sample response
	// [
	// {
	// "Path": "\\\\?\\scsi#disk\u0026ven_google\u0026prod_persistentdisk#4\u002621cb0360\u00260\u0026000100#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}",
	// "SerialNumber": " "
	// },
	// {
	// "Path": "\\\\?\\scsi#disk\u0026ven_msft\u0026prod_virtual_disk#2\u00261f4adffe\u00260\u0026000001#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}",
	// "SerialNumber": null
	// },
	cmd := "(Get-Disk \| Select Path, SerialNumber) \| ConvertTo-Json"
	out, err := runExec(cmd)
	if err != nil {
	return nil, fmt.Errorf("Could not query disk paths")
	}

	outString := string(out)
	disks := []Disk{}
	json.Unmarshal([]byte(outString), &disks)

	// GetVolumeStats - retrieves the volume stats for a given volume
	func (VolumeAPI) GetVolumeStats(volumeID string) (int64, int64, error) {
	// get the size and sizeRemaining for the volume
	cmd := fmt.Sprintf("(Get-Volume -UniqueId \"%s\" \| Select SizeRemaining,Size) \| ConvertTo-Json", volumeID)
	out, err := utils.RunPowershellCmd(cmd)

	if err != nil {
	return -1, -1, fmt.Errorf("error getting capacity and used size of volume. cmd: %s, output: %s, error: %v", cmd, string(out), err)
	}

	var getVolume map[string]int64
	outString := string(out)
	err = json.Unmarshal([]byte(outString), &getVolume)
	if err != nil {
	return -1, -1, fmt.Errorf("out %v outstring %v err %v", out, outString, err)
	}
	var volumeSizeRemaining int64
	var volumeSize int64

	volumeSize = getVolume["Size"]
	volumeSizeRemaining = getVolume["SizeRemaining"]

	volumeUsedSize := volumeSize - volumeSizeRemaining
	return volumeSize, volumeUsedSize, nil
	}

kubernetes-csi / csi-proxy Goto Github PK

csi-proxy's People

Contributors

Stargazers

Watchers

Forkers

csi-proxy's Issues

Bug Report

Tasks

Tasks

Tasks

Recommend Projects

Recommend Topics

Recommend Org