Comments (3)
See more information here:
from home-ops.
My steps to resolve:
# scale down deployment
k scale deployment.apps/nzbhydra2 --replicas 0
# export yaml for editing
k get pv (kubectl get pv --all-namespaces | grep "nzbhydra2-config" | awk '{print $1}') -o yaml > /tmp/pv.yaml
# change `controllerExpandSecretRef.name` value to `rook-csi-rbd-provisioner`
nano /tmp/pv.yaml
# replace
k replace --cascade=false --force -f /tmp/pv.yaml
# while the above is running do the following in a separate shell session
k edit pv (kubectl get pv --all-namespaces | grep "nzbhydra2-config" | awk '{print $1}')
# remove following code and save
finalizers:
- kubernetes.io/pv-protection
# now the k replace command should have finished
# scale back up
k scale deployment.apps/nzbhydra2 --replicas 1
# now give it a moment to resize and ignore the Claim Lost messages
k describe pvc/nzbhydra2-config
Events, after awhile (~10 minutes) the volume should be resized and the PVC should report the following.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Resizing 48m (x38 over 3h1m) external-resizer rook-ceph.rbd.csi.ceph.com External resizer is resizing volume pvc-3f7819a8-be4c-48c8-8cc0-e538675bde32
Warning VolumeResizeFailed 48m (x38 over 3h1m) external-resizer rook-ceph.rbd.csi.ceph.com resize volume pvc-3f7819a8-be4c-48c8-8cc0-e538675bde32 failed: error getting secret rook-ceph-csi in namespace rook-ceph: secrets "rook-ceph-csi" not found
Warning ExternalExpanding 45m volume_expand Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC.
Warning VolumeResizeFailed 12m (x21 over 40m) external-resizer rook-ceph.rbd.csi.ceph.com resize volume pvc-3f7819a8-be4c-48c8-8cc0-e538675bde32 failed: error getting secret rook-ceph-csi in namespace rook-ceph: secrets "rook-ceph-csi" not found
Warning ClaimLost 9m17s persistentvolume-controller Bound claim has lost its PersistentVolume. Data on the volume is lost!
Normal Resizing 52s (x23 over 40m) external-resizer rook-ceph.rbd.csi.ceph.com External resizer is resizing volume pvc-3f7819a8-be4c-48c8-8cc0-e538675bde32
Normal FileSystemResizeRequired 51s external-resizer rook-ceph.rbd.csi.ceph.com Require file system resize of volume on node
Normal FileSystemResizeSuccessful 8s kubelet, k3s-worker-d MountVolume.NodeExpandVolume succeeded for volume "pvc-3f7819a8-be4c-48c8-8cc0-e538675bde32"
from home-ops.
All PVCs have been updated to support resizing.
from home-ops.
Related Issues (20)
- Set an affinity for ceph mgr and mons to be on my master nodes HOT 2
- Create an application in Go to serve kube stats over a json endpoint HOT 1
- Link Checker Dashboard 🔗
- Publish my schemas in the cloud HOT 1
- Add `PushSecret` config for Ceph RGW buckets for Thanos HOT 4
- Consolidate method of pulling CLI tools in Github workflows HOT 1
- Update cloudflare terraform to use `cloudflare_ruleset` HOT 1
- Change lldap container to rootless HOT 1
- Replace kubelet-args with kubelet config file due to args being deprecated HOT 1
- Question on Ceph Version within Rook HOT 3
- kube-vip rbac manifest moved/removed HOT 1
- Replace redis with dragonflydb HOT 1
- Switch unpackerr to ghcr HOT 1
- No alerts are coming thru for failed helmreleases HOT 1
- Write a task to deploy code server for editing hass configs HOT 1
- Question on CloudNative-PG Upgrade HOT 2
- kromogo cluster_age_days calculation HOT 1
- Cluster bootstrap & upgrade nuances 🔴
- Re-evaluate storing `HelmRelease` resources in the `flux-system` namespace HOT 1
- Renovate Dashboard 🤖
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from home-ops.