Author: Matthew Cary (Google)
Kubernetes v1.27 graduated to beta a new policy mechanism for
StatefulSets that controls the lifetime of their
PersistentVolumeClaims (PVCs). The new PVC retention policy lets users specify if the PVCs generated from the
StatefulSet spec template should be automatically deleted or retrained when the
StatefulSet is deleted or replicas in the
StatefulSetare scaled down.
What problem does this solve?
StatefulSet spec can include
Pod and PVC templates. When a replica is first created, the Kubernetes control plane creates a PVC for that replica if one does not already exist. The behavior before the PVC retention policy was that the control plane never cleaned up the PVCs created for
StatefulSets - this was left up to the cluster administrator, or to some add-on automation that you’d have to find, check suitability, and deploy. The common pattern for managing PVCs, either manually or through tools such as Helm, is that the PVCs are tracked by the tool that manages them, with explicit lifecycle. Workflows that use
StatefulSets must determine on their own what PVCs are created by a
StatefulSet and what their lifecycle should be.
Before this new feature, when a StatefulSet-managed replica disappears, either because the
StatefulSet is reducing its replica count, or because its
StatefulSet is deleted, the PVC and its backing volume remains and must be manually deleted. While this behavior is appropriate when the data is critical, in many cases the persistent data in these PVCs is either temporary, or can be reconstructed from another source. In those cases, PVCs and their backing volumes remaining after their
StatefulSet or replicas have been deleted are not necessary, incur cost, and require manual cleanup.
StatefulSet PVC retention policy
StatefulSet PVC retention policy is used to control if and when PVCs created from a
volumeClaimTemplate are deleted. There are two contexts when this may occur.
The first context is when the
StatefulSet resource is deleted (which implies that all replicas are also deleted). This is controlled by the
whenDeleted policy. The second context, controlled by
whenScaled is when the
StatefulSet is scaled down, which removes some but not all of the replicas in a
StatefulSet. In both cases the policy can either be
Retain, where the corresponding PVCs are not touched, or
Delete, which means that PVCs are deleted. The deletion is done with a normalobject deletion, so that, for example, all retention policies for the underlying PV are respected.
This policy forms a matrix with four cases. I’ll walk through and give an example for each one.
Visit thedocumentation to see all the details.
Try it out! The
StatefulSetAutoDeletePVC feature gate is beta and enabled by default on cluster running Kubernetes 1.27. Create a
StatefulSet using the new policy, test it out and tell us what you think!
I'm very curious to see if this owner reference mechanism works well in practice. For example, I realized there is no mechanism in Kubernetes for knowing who set a reference, so it’s possible that the
StatefulSet controller may fight with custom controllers that set their own references. Fortunately, maintaining the existing retention behavior does not involve any new owner references, so default behavior will be compatible.
Please tag any issues you report with the label
sig/apps and assign them to Matthew Cary (@mattcary at GitHub).