Considerations for planning PXStore-V2 pool capacity and expansion

PX-StoreV2 pools have a predefined maximum size for storage pools that is set during the creation of the pool. This maximum size is based on the number and size of the drives included in the pool.

An analogy to help understand this concept is the inode count of a filesystem (fs). When a filesystem is created, the inode count (which determines the maximum number of files the filesystem can handle) is set and cannot be altered later. However, the filesystem's capacity can still be expanded as long as there are available inodes.

Similarly, in the case of pools, Portworx sets up the storage pool upper capacity based on the information available at the time of creation. Once created, the storage pool has a fixed upper limit, beyond which it cannot be expanded.

Starting with version 3.1.1, you can expand the PX-StoreV2 pool size by using the resize-drive operations, but you cannot add a new drive to the pool with the add-drive operation. As new drives cannot be added to the pool, there is an upper limit set for the pool capacity.

Here are the key considerations and recommendations for creating PX-StoreV2 pools:

1. Default maximum pool size: The default maximum size for PX-StoreV2 pools is set to 15TiB. This can be adjusted using the -max_thin_pool_size option to define the pool's maximum expandable size. For larger pool sizes, a bigger chunk size configuration is recommended.
2. Planning for pool capacity: If the maximum size a single drive can be resized to is 15TiB, then creating the pool with one disk is sufficient. For cases where a single disk's maximum resize limit is lower (e.g., 2TiB), it is recommended to create a pool with multiple disks (e.g., eight disks) to accommodate future growth within the 15TiB limit.
3. Adding new pools: When a pool reaches its maximum capacity, you can add a new pool to an existing node, by running the following command:
   • On-Premises: Use pxctl sv drive add -d /dev/sdm,/dev/sdn --newpool.
   • Cloud Environments: Use pxctl sv drive add -s type=io1,size=20 --newpool.
4. Considerations for vSphere VMDK backends in on-premises environments: When using vSphere VDMK disks for PX-StoreV2 pools, it's recommended that each datastore only allocates a single VDMK. Allocating multiple VDMKs to a single datastore could lead to the datastore filling up before the PX-StoreV2 pool reaches its designated capacity.