Skip to main content
Version: 26.1

Use CSI Topology

The CSI (Container Storage Interface) topology feature allows Kubernetes to make intelligent decisions about where to place workloads based on the underlying storage topology. This ensures that data is stored and accessed in the most efficient manner possible, reducing latency and improving performance. Follow the instructions on this page to enable CSI topology on an existing Kubernetes cluster using FlashArray and FlashBlade Direct Access volumes.

This feature ensures that storage is provisioned on the FlashArray/FlashBlade accessible from the same set of Kubernetes nodes where the application pods are running. This results in optimized storage placement, improved performance, and enhanced fault tolerance.

How CSI topology works​

The CSI topology feature uses topology labels to describe the placement of each FlashArray or FlashBlade resource. These labels can define attributes such as:

  • Region: Geographical area of the data center.
  • Zone: Logical subdivision within the region.
  • Datacenter or rack: Specific physical locations.
  • Provider: Cloud or infrastructure provider details.

When an application requests storage, the CSI driver evaluates the topology labels to determine the best storage resource close to the application pod. This proximity-based storage allocation:

  • Reduces latency: Ensures faster access to storage by minimizing network hops.
  • Enhances performance: Improves data throughput for applications.
  • Boosts fault tolerance: Supports replication across zones or regions, protecting against localized failures.

CSI topology for FlashArray and FlashBlade​

PX-CSI extends the capabilities of Kubernetes storage by integrating topology-aware provisioning for FlashArray and FlashBlade resources. It simplifies managing and optimizing storage for applications with key features such as:

  1. Node labeling with topology information:
    Kubernetes nodes can be labeled with attributes like region, zone, and rack. These labels guide the PX-CSI driver to provision storage on a FlashArray/FlashBlade that aligns with the application's location.

  2. Configuring topology in StorageCluster:
    During setup, you can include specific topology-related configurations:

    • Create a px-pure-secret to define topology labels for FlashArray or FlashBlade resources.

  3. Dynamic node affinity and volume binding modes:
    By defining node affinity in pod or StatefulSet specifications and using the WaitForFirstConsumer volume binding mode in your StorageClass, Portworx ensures that:

    • Storage is provisioned on the FlashArray/FlashBlade that satisfy topology requirements.
    • Applications benefit from reduced latency and better performance.

  4. Monitoring and performance tuning:
    PX-CSI provides tools to monitor storage performance and adjust configurations, such as the region and zone labels, to meet application needs dynamically.

Enable CSI topology on an existing cluster​

  1. Edit the cluster's StorageCluster object to include the following:

    csi:
      enabled: true
      topology:
        enabled: true

  2. Delete the existing px-pure-secret:

    kubectl delete secret --namespace <stc-namespace> px-pure-secret

  3. Create a new px-pure-secret using the following command:

    kubectl create secret generic px-pure-secret --namespace <stc-namespace> --from-file=<pure.json_file_path>

    Include Labels that specify the topology for each FlashArray. The keys must match a set of specific strings, but you can define your own values. For example:

    {
      "FlashArrays": [
        {
          "MgmtEndPoint": "<managementEndpoint>",
          "APIToken": "<apiToken>",
          "Labels": {
            "topology.portworx.io/zone": "zone-0",
            "topology.portworx.io/region": "region-0"
          }
        },
        {
          "MgmtEndPoint": "<managementEndpoint>",
          "APIToken": "<apiToken>",
          "Labels": {
            "topology.portworx.io/zone": "zone-1",
            "topology.portworx.io/region": "region-1"
          }
        }
      ]
    }

  4. Label your Kubernetes nodes with labels that correspond to the Labels from the previous step. For example:

    kubectl label node <nodeName> topology.portworx.io/zone=zone-0
    kubectl label node <nodeName> topology.portworx.io/region=region-0

  5. Delete the PX-CSI pods using the following command:

    kubectl delete pods -n portworx \
      -l 'app.kubernetes.io/component in (controller-plugin,node-plugin)'

    Wait until all pods are restarted and running on all nodes.

  6. Validate that topology is enabled on a node by describing csinode with the following command:

    kubectl describe csinode <node-name>
    Name:               <node-name>
    ...
    Spec:
      Drivers:
        pxd.portworx.com:
          Node ID:        <node-id>
          Topology Keys:  [topology.portworx.io/region topology.portworx.io/zone]

Topology-aware volume provisioning​

You can use the topology defined above by specifying the allowedTopologies in the StorageClass and nodeAffinity in the pod specification. The following example uses FlashArray block volumes. The steps are similar for FlashArray file services and FlashBlade file systems.

  1. Create a StorageClass with allowedTopologies:

    kind: StorageClass
    apiVersion: storage.k8s.io/v1
    metadata:
      name: zone1-sc
    provisioner: pxd.portworx.com
    parameters:
      backend: "pure_block"
      max_bandwidth: "10G"
      max_iops: "30000"
    volumeBindingMode: WaitForFirstConsumer
    allowedTopologies:
      - matchLabelExpressions:
          - key: topology.portworx.io/zone
            values:
              - zone-1
          - key: topology.portworx.io/region
            values:
              - region-1

  2. Create a PVC using the StorageClass created in the previous step:

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: zone1-pvc
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
      storageClassName: zone1-sc

  3. Specify the placement strategy by defining node affinity in your Pod or StatefulSet. For example:

    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: topology.portworx.io/zone
                operator: In
                values:
                - zone-1
              - key: topology.portworx.io/region
                operator: In
                values:
                - region-1

    The pod will be scheduled on a node that matches the topology defined in the pod node affinity, and the PVC will be provisioned on the array that matches the topology defined in the StorageClass.

Last updated on