Create sharedv4 PVCs


This document describes how to use Portworx sharedv4 (ReadWriteMany) volumes in your Kubernetes cluster.

Prerequisites

  • sharedv4 volumes must not be disabled on your cluster
  • NFS ports between nodes must be open

Provision a Sharedv4 Volume

Sharedv4 volumes are useful when you want multiple PODs to access the same PVC (volume) at the same time. They can use the same volume even if they are running on different hosts. They provide a global namespace and the semantics are POSIX compliant.

To increase fault tolerance, you can enable sharedv4 service volumes. With this feature enabled, every sharedv4 volume has a Kubernetes service associated with it. Sharedv4 service volumes expose the volume via a Kubernetes service IP. If the sharedv4 (NFS) server goes offline and requires a failover, application pods won’t need to restart.

Notes about Sharedv4 service volumes:

  • This is an early access feature.
  • Sharedv4 service volumes currently only support NFSv3.
  • Sharedv4 service volumes are intended for use within the Kubernetes cluster where the volume resides.

Known issue:

On failover, Applications may receive an error for non idempotent requests. For example, if an mkdir call is issued prior to failover, the client can resend it to the new server, which returns an EEXIST error if the directory was created by the first call.

Step1: Create Storage Class

  1. Create the following storageClass, specifying your own values for the following fields:

    • The metadata.name field with a name for your storageClass
    • The parameters.repl field with the replication factor you’d like to set
    • The sharedv4 field set to true
    • (Optional) The sharedv4_svc_type set to either ClusterIP or Loadbalancer

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
          name: px-sharedv4-sc
      provisioner: kubernetes.io/portworx-volume
      parameters:
        repl: "2"
        sharedv4: "true"
        sharedv4_svc_type: "ClusterIP"
  2. Apply the storageClass:

    kubectl apply -f examples/volumes/portworx/portworx-sharedv4-sc.yaml
  3. Verify the storage class is created:

    kubectl describe storageclass px-sharedv4-sc
    Name:	  	   px-sharedv4-sc
    IsDefaultClass:	   No
    Annotations:	   <none>
    Provisioner:	   kubernetes.io/portworx-volume
    Parameters:	   repl=2,sharedv4=true,sharedv4_svc_type=ClusterIP
    Events:			<none>

Step2: Create Persistent Volume Claim

Creating a ReadWriteMany persistent volume claim:

kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pvc.yaml

Example:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: px-sharedv4-pvc
  annotations:
    volume.beta.kubernetes.io/storage-class: px-sharedv4-sc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

Note the accessMode for this PVC is set to ReadWriteMany so the Kubernetes allows mounting this PVC on multiple pods.

Verifying persistent volume claim is created:

kubectl get pvc
NAME            STATUS    VOLUME                                   CAPACITY   ACCESSMODES   STORAGECLASS   AGE
px-sharedv4-pvc   Bound     pvc-a38996b3-76e9-11e7-9d47-080027b25cdf 10Gi       RWX           px-sharedv4-sc   12m

Step3: Create Pods which uses Persistent Volume Claim

We will start two pods which use the same shared volume.

Starting pod-1

kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pod-1.yaml

Example:

apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  containers:
  - name: test-container
    image: gcr.io/google_containers/test-webserver
    volumeMounts:
    - name: test-volume
      mountPath: /test-portworx-volume
  volumes:
  - name: test-volume
    persistentVolumeClaim:
      claimName: px-sharedv4-pvc

Starting pod-2

kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pod-2.yaml

Example:

apiVersion: v1
kind: Pod
metadata:
  name: pod2
spec:
  containers:
  - name: test-container
    image: gcr.io/google_containers/test-webserver
    volumeMounts:
    - name: test-volume
      mountPath: /test-portworx-volume
  volumes:
  - name: test-volume
    persistentVolumeClaim:
      claimName: px-sharedv4-pvc

Verifying pods are running:

kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE
pod1      1/1       Running   0          2m
pod2      1/1       Running   0          1m

Update a sharedv4 volume to sharedv4 service volume

Perform the following steps to convert a sharedv4 volume to use the new sharedv4 service feature:

  1. Detach the volume by scaling down the app.
  2. Run the following pxctl command:

    pxctl volume update --sharedv4_service_type=ClusterIP <volume>

Update a legacy shared volume to a sharedv4 volume

You can update an existing shared volume to use the new v4 protocol and convert it into a sharedv4 volume. Run the following pxctl command to update the volume setting

pxctl volume update --sharedv4=on <vol-name>
To access PV/PVCs with a non-root user refer here


Last edited: Thursday, Sep 30, 2021