Author: Matthew Cary (Google)
Kubernetes v1.27 graduated to beta a new policy mechanism forStatefulSets that controls the lifetime of their PersistentVolumeClaims (PVCs). The new PVC retention policy lets users specify if the PVCs generated from the StatefulSet spec template should be automatically deleted or retrained when the StatefulSet is deleted or replicas in the StatefulSetare scaled down.
What problem does this solve?
A StatefulSet spec can include Pod and PVC templates. When a replica is first created, the Kubernetes control plane creates a PVC for that replica if one does not already exist. The behavior before the PVC retention policy was that the control plane never cleaned up the PVCs created forStatefulSets - this was left up to the cluster administrator, or to some add-on automation that you’d have to find, check suitability, and deploy. The common pattern for managing PVCs, either manually or through tools such as Helm, is that the PVCs are tracked by the tool that manages them, with explicit lifecycle. Workflows that use StatefulSets must determine on their own what PVCs are created by a StatefulSet and what their lifecycle should be.
Before this new feature, when a StatefulSet-managed replica disappears, either because theStatefulSet is reducing its replica count, or because its StatefulSet is deleted, the PVC and its backing volume remains and must be manually deleted. While this behavior is appropriate when the data is critical, in many cases the persistent data in these PVCs is either temporary, or can be reconstructed from another source. In those cases, PVCs and their backing volumes remaining after their StatefulSet or replicas have been deleted are not necessary, incur cost, and require manual cleanup.
The new StatefulSet PVC retention policy
The new StatefulSet PVC retention policy is used to control if and when PVCs created from aStatefulSet’s volumeClaimTemplate are deleted. There are two contexts when this may occur.
The first context is when the StatefulSet resource is deleted (which implies that all replicas are also deleted). This is controlled by the whenDeleted policy. The second context, controlled bywhenScaled is when the StatefulSet is scaled down, which removes some but not all of the replicas in a StatefulSet. In both cases the policy can either be Retain, where the corresponding PVCs are not touched, or Delete, which means that PVCs are deleted. The deletion is done with a normalobject deletion, so that, for example, all retention policies for the underlying PV are respected.
This policy forms a matrix with four cases. I’ll walk through and give an example for each one.
whenDeletedandwhenScaledare bothRetain.whenDeletedisDeleteandwhenScaledisRetain.whenDeletedandwhenScaledare bothDelete.whenDeletedisRetainandwhenScaledisDelete.
Visit thedocumentation to see all the details.
What’s next?
Try it out! The StatefulSetAutoDeletePVC feature gate is beta and enabled by default on cluster running Kubernetes 1.27. Create a StatefulSet using the new policy, test it out and tell us what you think!
I'm very curious to see if this owner reference mechanism works well in practice. For example, I realized there is no mechanism in Kubernetes for knowing who set a reference, so it’s possible that the StatefulSet controller may fight with custom controllers that set their own references. Fortunately, maintaining the existing retention behavior does not involve any new owner references, so default behavior will be compatible.
Please tag any issues you report with the label sig/apps and assign them to Matthew Cary (@mattcary at GitHub).
Enjoy!
Top comments (0)