CSI topology for use in OpenShift with FlashArray cloud drives
The CSI topology feature for FlashArray Cloud Drives (FACD) allows applications to provision and use cloud drives only on FlashArrays that are in the same zone as a given node. This improves fault tolerance for replicas, performance, and manageability for large clusters.
Prerequisites
In order to use the CSI topology feature with a FlashArray Direct Access volume or FlashBlade Direct Access filesystem, you must meet the following prerequisites:
- Portworx version 3.0.0 or newer installed on a new Kubernetes cluster deployed on infrastructure that meets the minimum requirements for Portworx.
You cannot enable this feature on an existing Portworx installation.
-
Portworx Operator version 23.5.1 or newer.
-
One or more FlashArrays is reachable from worker nodes.
-
All nodes must have management access to all arrays.
-
All nodes must support iSCSI, FC, and NVMe technologies.
Enable CSI topology
To enable CSI topology, specify the topology.portworx.io/zone
label to describe the topology of each FlashArray.
Prepare your environment
Before installing Portworx, it is essential to label your Kubernetes nodes with the correct topology, specifically indicating the zone for each FlashArray. This step is crucial for proper management and access control, especially when working with multiple FlashArrays.
-
Label your Kubernetes nodes specifying the topology for each FlashArray, for example:
oc label node <nodeName> topology.portworx.io/zone=zone-1
oc label node <nodeName> topology.portworx.io/zone=zone-2 -
Create a
px-pure-secret
containing the information for your FlashArrays with thetopology.portworx.io/zone
label. For example:{
"FlashArrays": [
{
"MgmtEndPoint": "<managementEndpoint>",
"APIToken": "<apiToken>",
"Labels": {
"topology.portworx.io/zone": "zone-1"
}
},
{
"MgmtEndPoint": "<managementEndpoint>",
"APIToken": "<apiToken>",
"Labels": {
"topology.portworx.io/zone": "zone-2"
}
}
]
}
Generate the specs
-
Navigate to Portworx Central and log in, or create an account.
-
Select Portworx Enterprise from the product catalog and click Continue.
-
On the Product Line page, choose any option depending on which license you intend to use, then select Continue to start the spec generator.
-
Choose your Portworx version, platform, and click Customize at the bottom of the Summary section.
-
Specify your cluster information on the Basic, Storage and Network pages.
-
On the Customize page, specify the following in the Environment Variables tab:
- Name:
FACD_TOPOLOGY_ENABLED
- Value:
true
Also, ensure that the Enable CSI checkbox is selected in the Advance Settings tab.
Click Finish to create the specs. - Name:
Apply the specs and verify
-
Deploy the Operator:
oc apply -f 'https://install.portworx.com/<version-number>?comp=pxoperator'
serviceaccount/portworx-operator created
podsecuritypolicy.policy/px-operator created
clusterrole.rbac.authorization.k8s.io/portworx-operator created
clusterrolebinding.rbac.authorization.k8s.io/portworx-operator created
deployment.apps/portworx-operator created -
Deploy the StorageCluster:
oc apply -f 'https://install.portworx.com/<version-number>?operator=true&mc=false&kbver=&b=true&kd=type%3Dgp2%2Csize%3D150&s=%22type%3Dgp2%2Csize%3D150%22&c=px-cluster-XXXX-XXXX&eks=true&stork=true&csi=true&mon=true&tel=false&st=k8s&e==AWS_ACCESS_KEY_ID%3XXXX%2CAWS_SECRET_ACCESS_KEY%3XXXX&promop=true'
storagecluster.core.libopenstorage.org/px-cluster-XXXXXXX-XXXX-XXXX-XXXX-8dfd338e915b created
-
Verify if the cloud drives for a given node are provisioned on arrays that are in the same zone:
pxctl sv pool show
PX drive configuration:
Pool ID: 0
Type: Default
UUID: xxxxxxxx-xxxx-xxxx-xxxx-0d1797392d56
IO Priority: HIGH
Labels: topology.portworx.io/zone=zone-2,beta.kubernetes.io/arch=amd64,medium=STORAGE_MEDIUM_SSD,iopriority=HIGH,topology.portworx.io/region=region-2,kubernetes.io/hostname=dev-leather-forger-1,kubernetes.io/arch=amd64,kubernetes.io/os=linux,beta.kubernetes.io/os=linux
Size: 250 GiB
Status: Online
Has metadata: No
Balanced: Yes
....You will see that
Labels
flag is set for your volumes.