Skip to main content
Version: 3.2

Application I/O control in Rancher

Portworx volumes are created from a common storage pool and share their available IOPS (input/output operations per second) and bandwidth (measured in MBps) between all other provisioned Portworx volumes. If an application's volumes start consuming too many resources, it can become a "noisy neighbor" and reduce the I/O and network bandwidth available to other volumes in the pool.

The Portworx Application I/O Control feature provides you with a method for controlling an individual Portworx volume’s IOPS or bandwidth usage from within your cluster. You can define quotas through a StorageClass which prevent any applications from using excessive bandwidth. When you need to adjust the bandwidth, you can do so in real-time by updating the values in the StorageClass. The updated configuration will be applied to new volumes created after the change.

In addition to defining parameters in a StorageClass, you can set Application I/O Control values for volumes using the pxctl command line interface.

note

By default, this feature is not enabled.

Prerequisites​

  • Application I/O Control is supported for kernel 4.6 or higher.
  • Only the cgroup mount path /sys/fs/cgroup/blkio is supported. This is typically set by mount -t cgroup -o blkio none /sys/fs/cgroup/blkio. Uncommon distributions could have a different cgroup mount path.

Create volumes​

note
  • You do not need to restart the Portworx service, as the settings take effect immediately, even if the volume is already attached.
  • You can configure a maximum rate for IOPS (measured in input/output operations per second) and bandwidth (measured in MBps) simultaneously. However, you cannot set both IOPS and bandwidth parameters for the same operation type (read or write) at the same time. For example:
    • The configuration --max_iops 1024,off --max_bandwidth off,2 is valid because it specifies IOPS for read and bandwidth for write.
    • In contrast, the configuration --max_iops 1024,off --max_bandwidth 2,off will result in an error because it attempts to set both IOPS and bandwidth for the read operation. Here, 1024 IOPS refers to input/output operations per second, and 2 bandwidth corresponds to 2 MBps.
  • Once you set Application I/O Control values, you can check alerts or logs for messages indicating that Application I/O Control is not enabled. For example: IO throttling supported from kernel v4.6, this node has kernel v3.10 indicates that Application I/O Control is not supported.
  • IOPS might be misleading due to batching of small blocksize IOs into a larger one before IO reaches the pxd device, especially when using sharedV4 volumes. Bandwidth throttling is more consistent.

Apply using a StorageClass​

You can apply Application I/O Control through a StorageClass.

To configure IOPS throttling in a StorageClass, use the following format:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: portworx-io-profile
provisioner: pxd.portworx.com
parameters:
  repl: "2"
  io_profile: "db_remote"
  io_throttle_rd_iops: "1024"
  io_throttle_wr_iops: "1024"

To configure bandwidth throttling in a StorageClass, use the following format:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: portworx-io-profile
provisioner: pxd.portworx.com
parameters:
  repl: "2"
  io_profile: "db_remote"
  io_throttle_rd_bw: "10"
  io_throttle_wr_bw: "10"

You can also configure read IOPS throttling and write bandwidth throttling, or the inverse. For example:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: portworx-io-profile
provisioner: pxd.portworx.com
parameters:
  repl: "2"
  io_profile: "db_remote"
  io_throttle_rd_iops: "1024"
  io_throttle_wr_bw: "10"
note

The minimum value for rd_bw and wr_bw is 1, and the minimum value for rd_iops and wr_iops is 256.

Create using pxctl​

When you create volumes using pxctl, you can specify the --max_bandwidth and --max_iops flags to restrict the storage resources that volume uses:

pxctl volume create --max_bandwidth <ReadBW>,<WriteBW> <volname/volid> 
pxctl volume create --max_iops <ReadIOPS>,<WriteIOPS> <volname/volid>
note

The minimum value for --max_bandwidth is 1, and the minimum value for --max_iops is 256.

Update volumes using pxctl​

If you want to add Application I/O Control to an existing volume, or change its throttling values, you can do so using the --max_bandwidth and --max_iops flags with the pxctl volume update commands:

pxctl volume update --max_bandwidth <ReadBW>,<WriteBW> <volname/volid> 
pxctl volume update --max_iops <ReadIOPS>,<WriteIOPS> <volname/volid>
note

If you want to limit only read or write bandwidth/IOPS traffic, set the other value to off.

For example: Set read throttling for an existing volume to 1024 IOPS:

pxctl volume update --max_iops 1024,off exampleVolume

Remove throttling​

If you want to remove throttling from a volume, set the value of the --max_bandwidth and --max_iops flags to off for any throttling you wish to remove:

pxctl volume update --max_bandwidth off,<WriteBW> <volname/volid>
pxctl volume update --max_iops off,off <volname/volid>
Last updated on