Expand storage pools in IKS for Portworx-managed disks

If you're running on the cloud, consider automation into your decision for which pool resize approach you use. The pxctl service pool expand command allows you to perform resize operations without manually adding new drives or increasing drive capacity on your cluster.

When you enter the pxctl service pool expand command, Portworx uses your cloud provider's API to create new drives and attach them or to expand the existing drives with no further input from you.

You can control the pool expand operation by specifying which operation you want to use: resize-drive or add-drive, or you can specify auto to let Portworx determine the best way to resize your storage pools based on your cloud provider.

By default, each pool can have a maximum of 6 drives. If required, you can use the runtime option limit_drives_per_pool to change this value.

The maximum number of drives for the IBM Cloud platform, including boot drives, Portworx cloud drives, and any other drives, is 8.

Prerequisites

1. You must be running Portworx on IBM VPC Gen2 Platform (Portworx 2.11.0 or newer)

2. You must have the IBM Block CSI driver version 4.4 or newer. To check your version, run the following command:

ibmcloud ks cluster addon ls --cluster <cluster-id>

3. If you need to update the IBM CSI driver on your cluster, perform the following steps:

a. Remove the currrent version using the following command:

ibmcloud ks cluster addon disable vpc-block-csi-driver --cluster <cluster-id>

b. Enable cluster addon with --version 4.4:

ibmcloud ks cluster addon enable vpc-block-csi-driver --cluster <cluster-id> --version 4.4

c. Check that the correct version is present:

ibmcloud ks cluster addon ls --cluster <cluster-id>

OK
Name Version Health State Health Status
vpc-block-csi-driver 4.4* (4.3 default) - Enabling

Automatically expand a cloud-based pool

1. Run the following command to find the UUID for a pool:

pxctl service pool show

PX drive configuration:
Pool ID: 0
        UUID:  xxxxxxxx-xxxx-xxxx-xxxx-aef346e61d89
        IO Priority:  HIGH
        Labels:  iopriority=HIGH,medium=STORAGE_MEDIUM_SSD
        Size: 384 GiB
        Status: Online
        Has metadata:  Yes
        Balanced:  Yes
        Drives:
        1: /dev/sde, Total size 128 GiB, Online
        2: /dev/sdf, Total size 128 GiB, Online
        3: /dev/sdg, Total size 128 GiB, Online
        Cache Drives:
        No Cache drives found in this pool
Journal Device:
        1: /dev/sdc1, STORAGE_MEDIUM_SSD

2. Expand a cloud-based pool by entering the pxctl service pool expand command with the following options:

• The --operation option to specify the desired operation
• The --size option to set the minimum new size of the pool in GiB
• The --uid option to provide the ID of the pool you want to resize

pxctl service pool expand --operation auto --size 1000 --uid <pool-UUID>

For example:

pxctl service pool expand --operation auto --size 1000 --uid xxxxxxxx-xxxx-xxxx-xxxx-aef346e61d89

3. Once you submit the command, Portworx will expand the storage pool in the background. You can list the storage pools periodically to check if they have finished expansion.

   pxctl cluster provision-status

4. When invoked on the Portworx node where the storage pool resides, the following command provides detailed information about the status of the pool expand process.

   pxctl service pool show

Resize or add a new drive to a cloud-based pool

The auto operation automatically expands your pool capacity by increasing the pool size or adding the new drives to it. To perform a specific operation, replace auto with add-drive or resize-drive:

pxctl service pool expand --operation add-drive --uid <pool-ID> --size <new-storage-pool-size-in-GiB>

note

When running the command pxctl service pool expand --operation resize-drive --size <new-storage-pool-size-in-GiB> --uid <pool-ID> for specific nodes, you may encounter the following warning message in the output of kubectl describe pvc <name-of-px-cloud-drive-pvc>:

Waiting for user to (re-)start a pod to finish file system resize of volume on node.

This warning indicates that the file system resize process on the volume is pending until the pod is restarted. However, it does not impact any Portworx operations, as the storage pool reflects the correct size, and you can further expand the pool if necessary. Portworx continues to function normally, and you can safely ignore this warning.