Cloud backups for single PVCs in airgapped EKS

Summary and Key concepts

Summary:

This article provides a guide on creating cloud snapshots of Portworx volumes and restoring or cloning those snapshots for use in Kubernetes pods. It explains the prerequisites, including ensuring Stork is installed and configuring cloud secrets for authentication. The document covers how to create cloud snapshots using specific annotations and track their status, and it provides instructions for creating new Persistent Volume Claims (PVCs) from these snapshots using the Stork-provided stork-snapshot-sc storage class.

Kubernetes Concepts:

• PersistentVolumeClaim (PVC): A user request for storage in Kubernetes that can be snapshotted and cloned.
• Annotations: Metadata added to Kubernetes objects for managing snapshots, including cloud snapshots.
• Namespace: Provides isolation of resources like PVCs and snapshots.

Portworx Concepts:

• Stork: A Kubernetes extension for managing storage operations like cloud snapshots and restores.

• VolumeSnapshot: Represents a snapshot of a Portworx volume, used for creating backups and restoring data from the cloud.

• CloudSnap: The feature used for creating and managing cloud-based snapshots in Portworx.

This document will show you how to create cloud snapshots of Portworx volumes and how you can clone those snapshots to use them in pods.

note

You can restore these snapshots only in the cluster where the snapshots were taken.

Pre-requisites

Installing Stork

This requires that you already have Stork installed and running on your Kubernetes cluster. If you fetched the Portworx specs from the Portworx spec generator in Portworx Central and used the default options, Stork is already installed.

Configuring cloud secrets

To create cloud snapshots, one needs to setup secrets with Portworx which will get used to connect and authenticate with the configured cloud provider.

Follow instructions on the create and configure credentials section to setup secrets.

Creating cloud snapshots

The cloud snapshot method supports the following annotations:

• portworx/snapshot-type: Indicates the type of snapshot. For cloud snapshots, the value should be cloud.
• portworx/cloud-cred-id: (Optional) This specifies the credentials UUID if you have configured credentials for multiple cloud providers. In a situation where a single cloud provider is configured, this is not required.
• portworx.io/cloudsnap-incremental-count: This specifies the number of incremental cloud snapshots after which a full backup will be taken.

Example

In below example, we create a cloud snapshot for a PVC called mysql-data backed by a Portworx volume.

apiVersion: volumesnapshot.external-storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: mysql-snapshot
  namespace: default
  annotations:
    portworx/snapshot-type: cloud
spec:
  persistentVolumeClaimName: mysql-data

Once you apply the above object you can check the status of the snapshots using the kubectl get volumesnapshot.volumesnapshot.external-storage.k8s.io/ command with the name of your snapshot appended:

kubectl get volumesnapshot.volumesnapshot.external-storage.k8s.io/mysql-snapshot

NAME                             AGE
volumesnapshots/mysql-snapshot   2s

kubectl get volumesnapshotdatas

NAME                                                                            AGE
volumesnapshotdatas/k8s-volume-snapshot-xxxxxxxx-xxxx-xxxx-xxxx-5a34ec89e61c    1s

The creation of the volumesnapshotdatas object indicates that the snapshot has been created. If you describe the volumesnapshotdatas object you can see the Portworx Cloud Snapshot ID and the PVC for which the snapshot was created.

kubectl describe volumesnapshotdatas

Name:         k8s-volume-snapshot-xxxxxxxx-xxxx-xxxx-xxxx-5a34ec89e61c
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  volumesnapshot.external-storage.k8s.io/v1
Kind:         VolumeSnapshotData
Metadata:
  Cluster Name:
  Creation Timestamp:             2018-03-08T03:17:02Z
  Deletion Grace Period Seconds:  <nil>
  Deletion Timestamp:             <nil>
  Resource Version:               29989636
  Self Link:                      /apis/volumesnapshot.external-storage.k8s.io/v1/k8s-volume-snapshot-xxxxxxxx-xxxx-xxxx-xxxx-5a34ec89e61c
  UID:                            xxxxxxxx-xxxx-xxxx-xxxx-0214683e8447
Spec:
  Persistent Volume Ref:
    Kind:  PersistentVolume
    Name:  pvc-xxxxxxxx-xxxx-xxxx-xxxx-0214683e8447
  Portworx Volume:
    Snapshot Id:  xxxxxxxx-xxxx-xxxx-xxxx-33c5ab8d4d8e/149813028909420894-125009403033610837-incr
  Volume Snapshot Ref:
    Kind:  VolumeSnapshot
    Name:  default/mysql-snapshot-xxxxxxxx-xxxx-xxxx-xxxx-0214683e8447
Status:
  Conditions:
    Last Transition Time:  <nil>
    Message:
    Reason:
    Status:
    Type:
  Creation Timestamp:      <nil>
Events:                    <none>

Creating PVCs from cloud snapshots

When you install Stork, it also creates a storage class called stork-snapshot-sc. This storage class can be used to create PVCs from snapshots.

To create a PVC from a snapshot, you would add the snapshot.alpha.kubernetes.io/snapshot annotation to refer to the snapshot name.

For the above snapshot, the spec would like this:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-snap-clone
  annotations:
    snapshot.alpha.kubernetes.io/snapshot: mysql-snapshot
spec:
  accessModes:
     - ReadWriteOnce
  storageClassName: stork-snapshot-sc
  resources:
    requests:
      storage: 2Gi

Once you apply the above spec you will see a PVC created by Stork. This PVC will be backed by a Portworx volume clone of the snapshot created above.

kubectl get pvc

NAMESPACE   NAME                                   STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
default     mysql-data                             Bound     pvc-xxxxxxxx-xxxx-xxxx-xxxx-0214683e8447   2Gi        RWO            px-mysql-sc                 2d
default     mysql-snap-clone                       Bound     pvc-xxxxxxxx-xxxx-xxxx-xxxx-0214683e8447   2Gi        RWO            stork-snapshot-sc           2s