Managing Kubernetes Storage: Automating PV and PVC Cleanup

Kubernetes, the de facto orchestration system for containerized applications, offers robust solutions for managing storage through Persistent Volumes (PVs) and Persistent Volume Claims (PVCs). These resources ensure that storage persists beyond the lifecycle of individual pods, but managing them, especially cleaning up "Released" PVs and PVCs, can become a daunting task as your cluster grows. Today, I'll walk you through an automated solution for managing these resources, specifically focusing on cleaning up "Released" PVs and PVCs in your Kubernetes cluster.

Understanding PVs and PVCs

Before diving into the solution, let's briefly recap what PVs and PVCs are:

• Persistent Volumes (PVs): Cluster-wide storage units provisioned by an administrator. They are a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes.
• Persistent Volume Claims (PVCs): Requests for storage by a user. They allow a user to request storage resources without specific knowledge of the underlying storage infrastructure.

When a PVC is bound to a PV and then released, the PV can remain in a "Released" state, still holding onto the data but no longer bound to a PVC. Over time, these unclaimed resources can accumulate, necessitating a cleanup to free up storage space or to ensure compliance with data retention policies.

The Challenge of Cleanup

Manually tracking and cleaning up these "Released" PVs and their associated PVCs is time-consuming and error-prone. Automating this process saves time, reduces the likelihood of human error, and can be integrated into your regular maintenance routines or CI/CD pipelines.

Automated Cleanup Solution

To address this, I've developed a Python script that automates the listing and deletion of "Released" PVs and their associated PVCs within a specified namespace. This script utilizes the Kubernetes Python client to interact with your cluster's API, providing a straightforward command-line interface for managing these resources.

Key Features

• List "Released" PVs: Quickly view all "Released" PVs and their associated PVCs within a specified namespace.
• Safe Deletion: Option to delete "Released" PVs and their associated PVCs, with added error handling to manage resources already deleted or not found, preventing script failure and providing clear feedback.
• Namespace Specific: Operates within a specified namespace, allowing for targeted cleanup without affecting other areas of your cluster.

Prerequisites

• Python 3.6 or newer.
• Kubernetes Python client installed (pip install kubernetes).
• kubectl configured for your cluster.

Usage

1. Listing "Released" PVs by Namespace:

   python3 pv-cleanup.py <namespace>

This command lists all "Released" PVs and their associated PVCs within the specified namespace.

2. Deleting "Released" PVs and PVCs by Namespace:

   python3 pv-cleanup.py <namespace> --delete

This command deletes all "Released" PVs and their associated PVCs within the specified namespace. Use with caution, as this action is irreversible.

Safety and Considerations

• Data Backup: Always ensure data is backed up before deletion, especially if any PVs might contain critical information.
• Testing: Test the script in a non-production environment to ensure it meets your needs and understand its effects fully.

Conclusion

Automating the cleanup of "Released" PVs and PVCs in Kubernetes not only simplifies cluster management but also helps maintain a clean, efficient environment. By integrating such scripts into your operational routines, you can ensure that your Kubernetes storage remains well-organized and that resources are utilized effectively.

Remember, with great power comes great responsibility. Automate wisely, and happy Kubernetes'ing!

Here's the script:

import argparse
from kubernetes import client, config
from kubernetes.client.rest import ApiException

class KubernetesPVCPVManager:
    def __init__(self):
        # Load the kubeconfig file from the default location.
        config.load_kube_config()
        self.v1 = client.CoreV1Api()

    def list_released_pvs_by_namespace(self, namespace_filter):
        try:
            all_pvs = self.v1.list_persistent_volume()
            filtered_pvs = [
                pv for pv in all_pvs.items
                if pv.status.phase == "Released" and pv.spec.claim_ref and pv.spec.claim_ref.namespace == namespace_filter
            ]

            if filtered_pvs:
                print(f"Released PVs in {namespace_filter} namespace:")
                for pv in filtered_pvs:
                    claim_ref = pv.spec.claim_ref
                    print(f"- PV Name: {pv.metadata.name}, Capacity: {pv.spec.capacity['storage']}, StorageClass: {pv.spec.storage_class_name}, Claim: {claim_ref.namespace}/{claim_ref.name}")
            else:
                print(f"No released PVs found in {namespace_filter} namespace.")
        except ApiException as e:
            print(f"Exception when calling CoreV1Api->list_persistent_volume: {e}")

    def delete_released_pv_and_pvc(self, namespace_filter):
        try:
            all_pvs = self.v1.list_persistent_volume()
            for pv in all_pvs.items:
                if pv.status.phase == "Released" and pv.spec.claim_ref and pv.spec.claim_ref.namespace == namespace_filter:
                    pvc_name = pv.spec.claim_ref.name
                    try:
                        print(f"Attempting to delete PVC {pvc_name} in namespace {namespace_filter}")
                        self.v1.delete_namespaced_persistent_volume_claim(name=pvc_name, namespace=namespace_filter)
                        print(f"Successfully deleted PVC {pvc_name} in namespace {namespace_filter}")
                    except ApiException as e:
                        if e.status == 404:
                            print(f"PVC {pvc_name} in namespace {namespace_filter} not found. It might have already been deleted.")
                        else:
                            raise
                    try:
                        print(f"Attempting to delete PV {pv.metadata.name}")
                        self.v1.delete_persistent_volume(name=pv.metadata.name)
                        print(f"Successfully deleted PV {pv.metadata.name}")
                    except ApiException as e:
                        if e.status == 404:
                            print(f"PV {pv.metadata.name} not found. It might have already been deleted.")
                        else:
                            raise
        except ApiException as e:
            print(f"Exception when deleting PV or PVC: {e}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Manage released PVs and PVCs by namespace.')
    parser.add_argument('namespace', type=str, help='The namespace of the PVCs to manage.')
    parser.add_argument('--delete', action='store_true', help='Delete the released PVs and their associated PVCs in the specified namespace.')

    args = parser.parse_args()
    namespace_to_filter = args.namespace
    delete_option = args.delete

    manager = KubernetesPVCPVManager()
    if delete_option:
        manager.delete_released_pv_and_pvc(namespace_to_filter)
    else:
        manager.list_released_pvs_by_namespace(namespace_to_filter)