Giter VIP home page Giter VIP logo

Comments (4)

rootfs avatar rootfs commented on August 21, 2024

I see in the delete log:

rbdplugin: I1008 08:05:16.470638       1 controllerserver.go:163] deleting volume pvc-59a6fe14cad011e8
...
provisioner: I1008 08:05:26.470676       1 controller.go:107] GRPC error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
provisioner: E1008 08:05:26.470695       1 controller.go:1138] Deletion of volume "pvc-59a6fe14cad011e8" failed: rpc error: code = DeadlineExceeded desc = context deadline exceeded
provisioner: E1008 08:05:26.470797       1 goroutinemap.go:150] Operation for "delete-pvc-59a6fe14cad011e8[6b1db6e5-cad0-11e8-a12b-deadbeef0100]" failed. No retries permitted until 2018-10-08 08:05:26.970758008 +0000 UTC m=+1036.417161060 (durationBeforeRetry 500ms). Error: "rpc error: code = DeadlineExceeded desc = context deadline exceeded"
rbdplugin: I1008 08:05:32.984535       1 rbd_util.go:341] rbd: Deleting file for Volume: csi-rbd-622a252c-cad0-11e8-9112-deadbeef0101 at: /var/lib/kubelet/plugins/csi-rbdplugin/controller resulting path: /var/lib/kubelet/plugins/csi-rbdplugin/controller/csi-rbd-622a252c-cad0-11e8-9112-deadbeef0101.json

it seems the provisioner timed out before rbd delete finished.

for rbd driver, deletion started without validating if the volume ID was still there. For sanity, we need to check local cache first around here https://github.com/ceph/ceph-csi/blob/master/pkg/rbd/controllerserver.go#L163

@pohly can you help with the sanity check? thanks

from ceph-csi.

pohly avatar pohly commented on August 21, 2024

@rootfs: I agree with your analysis. To make the call idempotent, it would have to return success when the volume info cannot be loaded.

But I am still worried that this local .json file is needed at all. Do you agree that this can only work across cluster and node restarts if the provisioner always runs on the same host? Any plans to address this?

from ceph-csi.

gman0 avatar gman0 commented on August 21, 2024

@pohly we already have an issue for this #66

from ceph-csi.

rootfs avatar rootfs commented on August 21, 2024

@pohly
agree, local json is persistent and won't be crash resistent.
as discussed with @gman0 , for kubernetes use case, we should consider a more robust cluster level storage for meta info. A configmap is a way to go for that.

@pohly if you are volunteering for #66, that'll be a great contribution, thanks.

from ceph-csi.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.