There have been at least a few issues on Kubernetes Community Operator related to PVC expansion, yet this is not documented anywhere.
There have been at least a few issues on Kubernetes Community Operator related to PVC expansion, yet this is not documented anywhere.
Looking at closed issues, one may get an idea that this operation is impossible, but with the latest CSI it should be - it requires some handcrafting. On the other side Enterprise operator has this covered here - https://www.mongodb.com/docs/kubernetes-operator/stable/tutorial/resize-pv-storage/. I suggest writing similar how-to for the Community operator.
My currently tested workflow for volume expansion is similar to this explained on the page above, but as for my knowledge Community operator doesn't allow this operation - as it will recreate StatefulSet when you delete, and there's no way for pause reconciliation of MongoDBCommunity. So it requires you to scale down operator deployment for the time of expansion.
So it looks like this:
1. Scale down mongodb-kubernetes-operator to 0
2. Dump current mongodb StatefulSet specs to file
3. Delete StatefulSet leaving pods (--cascade=orphan)
4. Modify saved StatefulSet specs in the file and update volumeClaimTemplates to match your new size requirements
And here it splits, as I'm using Longhorn, and it needs some time to detach and resize the volume. Because of that, using rollout - as described on the Enterprise operator hasn't worked. So for possibly any other CSI plugin, you should continue as described on the page:
- Patch PVCs with the new size
- Re-create StatefulSet with new specs for
volumeClaimTemplates - Rollout restart StatefulSet
But in my case, it was like this:
- Delete one pod (secondary RS members first, primary last - as usual)
- Wait for PV to be in a detached state
- Patch PVC
- Wait for PV to resize
- Re-create StatefulSet with new specs for
volumeClaimTemplates - Wait for it to reconcile
- Delete StatefulSet, leaving pods again
- Repeat the above steps for any other pods in your StatefulSet
And finally, make changes to MongoDBCommunity specs to match those in the manually created StatefulSet and scale-up operator.
Sebastian
shared this idea
Planned - no fixed date but likely complete within Q2.