Write Zeroes for RWX Block Volumes

When importing or cloning a virtual machine (VM) image, the source image can contain large unused regions, or holes, that represent empty disk space. During the copy operation, these regions can be sent to the destination block device as zero-write operations. Writing these zeroed regions consumes network bandwidth, compute resources, and storage capacity even though they do not contain application data.

Write Zeroes optimizes these operations by avoiding physical writes of zeroed regions to thin-provisioned storage. When Portworx Enterprise receives a supported kernel REQ_OP_WRITE_ZEROES request, it converts the request into a discard (unmap) operation. Instead of writing blocks of zeros, Portworx leaves the corresponding storage region unallocated.

This optimization is particularly useful for raw block (RWX Block) volumes used as KubeVirt VM disks, including VM image imports and cloning operations such as VMware vSphere migrations with Forklift and Containerized Data Importer (CDI) imports.

Write Zeroes provides the following benefits:

• Reduced network I/O: Avoids transferring zeroed data to the underlying storage.
• Reduced compute resource consumption: Avoids processing unnecessary zero-write operations on the host.
• Faster VM imports and cloning: Reduces the amount of data that must be processed and written during image operations.
• Lower storage consumption: Keeps zeroed regions unallocated on thin-provisioned pools instead of consuming physical storage capacity.

note

The first write to a newly allocated block can incur additional latency because thin pool zeroing initializes the block with zeros during allocation.

How Write Zeroes preserves data

Write Zeroes converts only kernel-generated REQ_OP_WRITE_ZEROES operations into discard requests. Applications that explicitly write zero-filled blocks generate standard REQ_OP_WRITE operations, which Portworx preserves.

Discarded blocks must read back as zeros. Therefore, Write Zeroes requires thin pool zeroing on every storage pool that hosts a replica of the volume.

Prerequisites

Write Zeroes becomes active for a volume only when all of the following requirements are met:

• All Portworx nodes run Portworx Enterprise 3.7.0 or later.
• The PX-FUSE driver supports the PXD_FEATURE_WRITE_ZEROES capability.
• The Linux kernel supports REQ_OP_WRITE_ZEROES (Linux 4.10 or later).
• The volume uses the native I/O path.
• Thin pool zeroing is enabled on every pool that hosts a replica of the volume.
• Discard operations are enabled for the volume.

important

If the PX-FUSE driver does not support PXD_FEATURE_WRITE_ZEROES, Portworx does not enable Write Zeroes for the volume and records the reason in the Portworx logs.

Limitations

Write Zeroes has the following limitations:

• Write Zeroes is supported only on new Portworx Enterprise deployments.
• Write Zeroes is not supported for FlashArray Direct Access (FADA) or FlashBlade Direct Access (FBDA) volumes.
• Write Zeroes is not supported on the fast path. While Write Zeroes is enabled for a volume, Portworx prevents the volume from being promoted from the native I/O path to the fast path.
• Write Zeroes is not supported on volumes that have discard operations disabled (nodiscard).
• Write Zeroes does not reclaim capacity on the underlying FlashArray.
• Thin pool zeroing cannot be disabled after it is enabled for a storage pool.

Enable Write Zeroes

Write Zeroes has two levels of configuration:

• Cluster level: Enables Write Zeroes across the cluster and configures storage pools for thin pool zeroing.
• Volume level: Enables Write Zeroes for individual raw block volumes.

You must enable both settings for Write Zeroes to become active on a volume.

Enable Write Zeroes for the cluster

Enable Write Zeroes through the runtimeOptions field in the StorageCluster specification:

spec:
  runtimeOptions:
    enable_write_zeroes: "1"

The following table describes the runtime option:

Parameter	Description	Default value
enable_write_zeroes	Enables cluster-wide Write Zeroes support for raw block (RWX Block) volumes and enables thin pool zeroing for storage pools on all nodes. Set the value to "1" to enable Write Zeroes.	0 (disabled)

important

• Enabling enable_write_zeroes configures all storage pools for thin pool zeroing and triggers a rolling restart of the Portworx nodes. Thin pool zeroing cannot be reverted after it is enabled.
• After you enable Write Zeroes at the cluster level, you cannot disable it at the cluster level.

Enable Write Zeroes for a volume

To enable Write Zeroes for newly provisioned volumes, set writezero to "true" in the StorageClass:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: px-writezero
provisioner: pxd.portworx.com
parameters:
  repl: "2"
  writezero: "true"

Alternatively, create a Portworx volume with Write Zeroes enabled:

pxctl volume create <volume-name> --writezero

To disable Write Zeroes for a volume, run:

pxctl volume update <volume-name> --writezero=off

note

Changes to the volume-level setting take effect the next time the volume is attached. If you disable Write Zeroes while the volume is attached, Write Zeroes remains active until you detach and reattach the volume.

Enable Write Zeroes for an existing volume

Existing volumes have Write Zeroes disabled by default.

To enable Write Zeroes for an existing volume:

1. Ensure that Write Zeroes is enabled at the cluster level. For more information, see Enable Write Zeroes for the cluster.

2. Enable Write Zeroes for the volume:

   pxctl volume update <volume-name> --writezero=on

3. Detach and reattach the volume for the change to take effect.

Verify Write Zeroes

To verify that the attached block device advertises Write Zeroes support, run the following command on the node where the volume is attached:

cat /sys/block/pxd\!pxd<id>/queue/write_zeroes_max_bytes

Replace <id> with the PXD device ID.

A nonzero value indicates that Write Zeroes is active for the block device.

To inspect the volume configuration and I/O path, run:

pxctl volume inspect <volume-name>

To monitor discard operations on the block device, run:

pxctl service perf report