Provision storage pool

Portworx creates one or more storage pools by grouping together homogenous drives that are of the same size, medium type and drive type (for cloud drives).

note

This document summarizes various configurable parameters as an Early Access(EA) available to provision storage pools and also covers higher capacity provisioning for PX-StoreV2 pools. It is applicable for custom storage pool creation. For default setup, do not configure these parameters and use the default options provided in the spec generation by Portworx.

How to provision storage pools

Pools can be provisioned either while installing Portworx or post Portworx installation.

Pool provisioning during Portworx installation

• On-Prem Installation: Provide an explicit list of drives to be used for creating storage pools.

• Cloud Managed Installation: Specify the number of drives and the capacity of each drive for the selected drive type. The spec generator offers a default configuration with an initial drive size and count optimized for pool scalability through disk resizing.

note

By default, Portworx allows PX-StoreV2 storage pools to grow up to 15 TiB in capacity. PX-StoreV2 storage pools can scale up to 480 TiB through resize disk expansion and additional settings applied during pool creation. PX-StoreV1 has the additional option of adding more disks to the pool.

Pool provisioning post Portworx installation

New pools can be created post Portworx installation using pxctl CLI service mode utilities. The pxctl service command provides the drive add option to add storage. The command accepts a list of drives (on-prem) or a drive count (cloud environment) to create pools with multiple drives. For cloud environments, if a drive count is not specified, Portworx would select a default drive count.

What are Portworx storage pool configurable parameters

The following configurable parameters allow Portworx pool provisioning with custom configuration. All configurable parameters are available as cluster runtime options that could be specified:

• Using StorageCluster spec during Portworx installation
• Using pxctl cluster runtime options post Portworx installation
• Using pxctl for new storage pool post Portworx installation

note

The total number of drives that can be attached to a node is limited and varies by cloud provider. Refer to your cloud provider's drive limits per node and the relevant cluster option in the Expand Your Storage Pool Size with Disks Managed by Portworx document.

limit_drives_per_pool

note

This parameter will be available with General Availability(GA) release and is not part of the Early Access(EA).

limit_drives_per_pool specifies the maximum number of drives per pool and is applicable for both PX-StoreV1 and PX-StoreV2. The supported range is 1 to 32 and the default value is set to 6.

1. For pool creation during Portworx installation, if the limit exceeds this number, the drives would be split into multiple pools. For example, if limit_drives_per_pool is 6 and number of drives of the same type specified is 8, one pool would have 6 drives and the other pool would have 2 drives.

2. For pool creation after Portworx installation using CLI, this limit can simply be overridden by specifying drives-count during newpool creation. This parameter is not exercised during CLI newpool creation.

limit_pools_count

This parameter specifies the maximum number of pools per node and is applicable for both PX-StoreV1 and PX-StoreV2. The supported range is 1 to 32 and the default set to 8.

max_pool_size_tb

This parameter specifies the maximum capacity that the PX-StoreV2 pool can grow later after creation(in TiB units). This parameter is only applicable for PX-StoreV2. If the value is not specified, Portworx selects a default value of 15 TiB. If you want a customized value of max pool size (non-default), it has to be explicitly specified for PX-StoreV2 pools.

How to update Portworx storage pool configurable parameters

Portworx provides multiple methods at different levels of installation to update the storage pool parameters.

Method 1: Using StorageCluster spec during Portworx installation

Pass the following annotations to the StorageCluster spec manually and set the value based on the requirement of parameters. There are two options to add the annotations:

• Option 1: portworx.io/misc-args: --rt_opts "limit_pools_count=2"
• Option 2:

     runtimeOptions:
        limit_pools_count=2

After updating the StorageCluster spec, apply them to the cluster to configure Portworx storage pool. In the above examples, the value for limit_pools_count is set as 2. Restart the Portworx service after changes are made through STC to allow the new settings to take effect.

Method 2: Using pxctl cluster runtime options post Portworx installation

Pool configuration options are runtime-configurable via cluster settings. Here are examples of how to apply updates using runtime-options.

pxctl cluster options update --runtime-options "limit_drives_per_pool=4"  
pxctl cluster options update --runtime-options "limit_pools_count=10" 
pxctl cluster options update --runtime-options "max_pool_size_tb=60"
pxctl cluster options update --runtime-options "limit_drives_per_pool=4,limit_pools_count=10,max_pool_size_tb=60" 

Method 3: Using pxctl for new storage pool post Portworx installation

The pxctl service command provides the drive add option to add storage. These options are the same for PX-StoreV1 and PX-StoreV2. CLI cluster runtime options do not need Portworx service restart to take effect.

note

The cluster runtime option settings need to be reset back to respective defaults which is limit_drives_per_pool should be set to 6 and max_pool_size_tb should be set to 0 after pool creation, otherwise the set values will continue to be in use across the cluster.

How to calculate initial number of drives in pool for cloud drives

If you need a pool with a total capacity of X TiB and the maximum drive size is Y TiB, calculate the required number of drives (N) as follows:

Number of drives(N) = ceil(Desired pool capacity / Maximum drive size)

ceil function rounds a number up to the nearest integer.

Example: You need a pool with 10 TiB of total capacity and maximum drive size is 1.92 TiB.

Number of drives = ceil(10 / 1.92)  
  = ceil(5.2083)  
  = 6

If you do not know the maximum pool capacity while creating the pool, let Portworx select the default value for the number of drives in the pool.

PX-StoreV2 capacity provisioning

PX-StoreV2 pools can scale roughly up to 480 TiB in capacity through resize disk expansion. By default, Portworx makes selections such that the pool can grow up to 15 TiB in capacity.

For provisioning higher capacity pools, you need to specify the max_pool_size_tb during pool creation to allow it to expand to the specified size. You also need to select an appropriate number of drives in the pool so that they can reach the specified max size.

For example, if you want the Portworx pool to be 240 TiB in capacity and if each drive can resize up to maximum of 45 TiB, you need at least 6 disks in the pool.

Default selection of number of drives in pool for cloud drives

Default selection of drives count based on the backend cloud provider. The default selection applies during any new pool creation.

1. For new pool creation during new install, spec generator would default below values unless you make a selection
2. For new pool creation post Portworx install, if drives count is not specified in the CLI creation of new pool (pxctl service drive add --newpool), below values would be used as default.

Sr no	Cloud provider	Default number of drives
1	Vsphere	4
2	AWS	4
3	Azure	4
4	Pure	1
5	GKE	4
6	All others	4

Use cases for creating pools on cloud drive environment

Pool creation during new Portworx installation

1. You do not have any specific requirements for pool capacity:

   Solution: Spec generator will make default selections such that the pool would reach specified size. This is suggested where there are non custom requirements for pool capacity.

2. You require a single storage pool with 150 TiB capacity.:

   Solution: Based on cloud drive type, find out the maximum size it can expand up to. Find out the number of drives required in the pool by calculating as discussed in section above. Select the number of drives in the spec generator. If the calculated number of drives in the pool exceeds 6, specify limit_drives_per_pool value as the number of drives through runc runtime options while installing Portworx. Additionally specify max_pool_size_tb value as 150 through runc runtime options for PX-StoreV2 pools only. Do not specify max_pool_size_tb for PX-StoreV1 pools.

   Reset the cluster runtime options after pool creation to respective defaults.

3. You require multiple pools, each of capacity of 200 TiB with homogenous drives:

   Solution: Based on cloud drive type, find out the maximum size it can expand up to. Find out the number of drives required in the pool by calculating as discussed in section above. Select twice the number of drives (as there are two pools). Specify limit_drives_per_pool value as the number of drives in a single pool through runc runtime options while installing PX. Additionally specify max_pool_size_tb value as 200 through runc runtime options for PX-StoreV2 pools. Do not specify max_pool_size_tb for PX-StoreV1 pools.

   Reset the cluster runtime options after pool creation to respective defaults.

4. You require multiple pools of different capacity of 100 TiB and 200 TiB with homogenous drives:

   Solution: In this case, it is advisable to create one pool during installation using solution provided in use case 2 and another pool after installation using method mentioned in post installation use case 2.

5. You require two pools, one of capacity 100 TiB and the other of 200 TiB with heterogenous drives:

   Solution: If both the pools have different capacity drives, calculate the number of drives for both pools and if one or both of them goes beyond 6, set the limit_drives_per_pool value to the higher value and install PX. Additionally specify max_pool_size_tb value as 200 through runc runtime options for PX-StoreV2 pools. Do not specify max_pool_size_tb for PX-StoreV1 pools.

   Reset the cluster runtime options after pool creation to respective defaults.

Pool creation post Portworx install

1. You do not have any specific requirements for pool capacity:

   Solution: Create newpool using the pxctl service command as shown below:

   pxctl service drive add --newpool -s <type=y,size=x>

   Additional optional argument --drives-count is available to select the number of drives in the pool. If no selection is made, PX would pick the default number of drives as indicated in section above.

2. You require a single pool of 180 TiB capacity:

   Solution: Create newpool using the following command:

   pxctl service drive add --newpool -s <type=y,size=x> --max-pool-size-tb 180 --drives-count n

   The max-pool-size-tb needs to be specified only for PX-StoreV2 pools. Additional optional argument --drives-count is available to select the number of drives in the pool. If no selection is made, PX would pick the default drive count based on the table in the above section. The drives count needs to be calculated as follows and at least that many drives need to be specified.

   Number of drives(n) = ceil(Desired pool capacity / Maximum drive size).

   If a lesser number of drives are specified, the pool creation would fail with an appropriate error message suggesting you to select correct configurable values.

   Note: The --max-pool-size-tb is an optional CLI argument to specify max pool size only for this new pool creation. Alternatively, this can also be set via the cluster runtime option settings as well discussed in the previous sections, but you need to reset the cluster runtime option values back to respective defaults after pool creation completes, otherwise the new setting would apply for all future new pool creations on the cluster. Do not specify max_pool_size_tb for PX-StoreV1 pools.

Troubleshooting

Following are common misconfigurations during new pool creation:

1. The number of drives specified by you are insufficient to reach the specified value of max pool size (PX-StoreV2 only). Based on pre or post installation, choose one of the following:

   1. For pool creation during new Portworx installations: Configure the limit_drives_per_pool and max_pool_size_tb values correctly as detailed in this document and create the pool again.
   2. For pool creation post Portworx installation: Configure the drives-count and max-pool-size-tb CLI parameters (pxctl service drive add --newpool) correctly as detailed in this document and create the pool again

2. The number of drives specified exceeds the maximum number of cloud drives that could be attached on the node. In that case, as a fix, create a new pool on a different node.

3. The pool gets created with misconfigured values. To fix this, check if there is no data on the pool, then delete the pool and re-create it again. Reach out to technical support in case of further assistance.