Portworx
PORTWORX.COM

Create and use cloud snapshots

This document shows how you can create cloud snapshots of Portworx volumes and how you can clone those snapshots to use them in pods.

NOTE: You cannot use an older version of Portworx to restore a cloud snapshot created with a newer one. For example, if you’re running Portworx 2.1, you can’t restore a cloud snapshot created with Portworx 2.2.

Back up a volume and restore it to the same Portworx cluster

This section shows how you can back up a volume and restore it to the same Portworx cluster.

Prerequisites

  • This requires that you already have Stork installed and running on your Kubernetes cluster. If you fetched the Portworx specs from the Portworx spec generator in PX-Central and used the default options, Stork is already installed.
  • Cloud snapshots using below method is supported in Portworx version 1.4 and above.
  • Cloud snapshots (for aggregated volumes) using below method is supported in Portworx version 2.0 and above.

Configuring cloud secrets

To create cloud snapshots, one needs to setup secrets with Portworx which will get used to connect and authenticate with the configured cloud provider.

Follow instructions on the create and configure credentials section to setup secrets.

Create cloud snapshots

With cloud snapshots, you can either snapshot individual PVCs one by one or snapshot a group of PVCs.

Restore cloud snapshots

Once you’ve created a cloud snapshot, you can restore it to a new PVC or the original PVC.

Restore a cloud snapshot to a new PVC

When you install Stork, it also creates a storage class called stork-snapshot-sc. This storage class can be used to create PVCs from snapshots.

To create a PVC from a snapshot, you would add the snapshot.alpha.kubernetes.io/snapshot annotation to refer to the snapshot name. If the snapshot exists in another namespace, the snapshot namespace should be specified with the stork.libopenstorage.org/snapshot-source-namespace annotation in the PVC.

Note that the storageClassName needs to be the Stork StorageClass stork-snapshot-sc as in the example below.

The following spec restores a PVC from the snapshot in the example above:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-snap-clone
  annotations:
    snapshot.alpha.kubernetes.io/snapshot: mysql-snapshot
spec:
  accessModes:
     - ReadWriteOnce
  storageClassName: stork-snapshot-sc
  resources:
    requests:
      storage: 2Gi

Once you apply the above spec, you will see a PVC created by Stork. This PVC will be backed by a Portworx volume clone of the snapshot created above.

kubectl get pvc
NAMESPACE   NAME                                   STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
default     mysql-data                             Bound     pvc-f782bf5c-20e7-11e8-931d-0214683e8447   2Gi        RWO            px-mysql-sc                 2d
default     mysql-snap-clone                       Bound     pvc-05d3ce48-2280-11e8-98cc-0214683e8447   2Gi        RWO            stork-snapshot-sc           2s

Restore a cloud snapshot to the original PVC

When you perform an in-place restore to a PVC, Stork takes the pods using that PVC offline, restores the volume from the snapshot, then brings the pods back online.

NOTE: In-place restore using VolumeSnapshotRestore works only for applications deployed using the stork scheduler. If you’re not using the Stork scheduler, Portworx displays the following error when describing the VolumeSnapshotRestore resource:

Events:
  Type     Reason  Age               From   Message
  ----     ------  ----              ----   -------
  Warning  Failed  5s (x2 over 15s)  stork  application not scheduled by stork scheduler

  1. Create a VolumeSnapshotRestore YAML file specifying the following:

    • apiVersion as stork.libopenstorage.org/v1alpha1
    • kind as VolumeSnapshotRestore
    • metadata.name with the name of the object that performs the restore
    • metadata.namespace with the name of the target namespace
    • spec.sourceName with the name of the snapshot you want to restore
    • spec.sourceNamespace with the namespace in which the snapshot resides

    The following example restores data from a snapshot called mysql-snapshot which was created in the mysql-snap-restore-splocal namespace to a PVC called mysql-snap-inrestore in the default namespace:

     apiVersion: stork.libopenstorage.org/v1alpha1
 kind: VolumeSnapshotRestore
 metadata:
   name: mysql-snap-inrestore
   namespace: default
 spec:
   sourceName: mysql-snapshot
   sourceNamespace: mysql-snap-restore-splocal

  2. Place the spec into a file called mysql-cloud-snapshot-restore.yaml and apply it:

     kubectl apply -f mysql-cloud-snapshot-restore.yaml

  3. You can enter the following command to see the status of the restore process:

     storkctl get volumesnapshotrestore
     NAME                   SOURCE-SNAPSHOT   SOURCE-SNAPSHOT-NAMESPACE   STATUS          VOLUMES   CREATED
 mysql-snap-inrestore   mysql-snapshot    default                     Successful      1         23 Sep 19 21:55 EDT

    You can also use the kubectl describe command to retrieve more detailed information about the status of the restore process.

    Example:

      kubectl describe volumesnapshotrestore mysql-snap-inrestore
      Name:         mysql-snap-inrestore
  Namespace:    default
  Labels:       <none>
  Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                  {"apiVersion":"stork.libopenstorage.org/v1alpha1","kind":"VolumeSnapshotRestore","metadata":{"annotations":{},"name":"mysql-snap-inrestore...
  API Version:  stork.libopenstorage.org/v1alpha1
  Kind:         VolumeSnapshotRestore
  Metadata:
    Creation Timestamp:  2019-09-23T17:24:30Z
    Generation:          5
    Resource Version:    904014
    Self Link:           /apis/stork.libopenstorage.org/v1alpha1/namespaces/default/volumesnapshotrestores/mysql-snap-inrestore
    UID:                 00474a5c-de27-11e9-986b-000c295d6364
  Spec:
    Group Snapshot:    false
    Source Name:       mysql-snapshot
    Source Namespace:  default
  Status:
    Status:  Successful
    Volumes:
      Namespace:  default
      Pvc:        mysql-data
      Reason:     Restore is successful
      Snapshot:   k8s-volume-snapshot-cb909cf9-de26-11e9-ad56-320ff611f4ca
      Status:     Successful
      Volume:     pvc-8b996a17-de26-11e9-986b-000c295d6364
  Events:
    Type    Reason      Age   From   Message
    ----    ------      ----  ----   -------
    Normal  Successful  0s    stork  Snapshot in-Place  Restore completed

References

Back up a volume and restore it to a different Portworx cluster

This section shows how you can back up a volume and restore it to a different Portworx cluster using the pxctl command-line utility.

Prerequisites

Before you can back up and restore a volume to a different Portworx cluster, you must meet the following prerequisites:

  • Two running Portworx clusters . Refer to the Installation page for details about how to install Portworx.
  • An object store. Cloud snapshots work with Amazon S3, Azure Blob, Google Cloud Storage, or any S3 compatible object store. If you don’t have an object store, Portworx, Inc. recommends using MinIO. See the MinIO Quickstart Guide page for details about installing MinIO.
  • A secret store provider. Refer to the Secret store management page for details about configuring a secret store provider.

Create your cloud snapshot credentials on the source cluster

The options you use to create your cloud snapshot credentials differ based on which secret store provider you use. The steps in this document describe AWS KMS, but you can find instructions for creating other credentials in the CLI reference.

  1. Enter the pxctl credentials create command, specifying the following:

    • The --provider flag with the name of the cloud provider (s3).
    • The --s3-access-key flag with your secret access key
    • The --s3-secret-key flag with your access key ID
    • The --s3-region flag with the name of the S3 region (us-east-1)
    • The --s3-endpoint flag with the name of the endpoint (s3.amazonaws.com)
    • The optional --s3-storage-class flag with either the STANDARD or STANDARD-IA value, depending on which storage class you prefer
    • The name of your cloud credentials

    Example:

      pxctl credentials create --provider s3 --s3-access-key <YOUR_ACCESS_KEY> --s3-secret-key <YOUR_SECRET_KEY> --s3-region us-east-1 --s3-endpoint <YOUR_ENDPOINT> --s3-storage-class <YOUR_STORAGE_CLASS> <YOUR_SOURCE_S3_CRED>
      Credentials created successfully, UUIDU0d9847d6-786f-4ed8- b263-5cde5a5a12f5

  2. You can validate your cloud snapshot credentials by entering the pxctl credentials validate command followed by the name of your cloud credentials:

      pxctl cred validate <YOUR_SOURCE_S3_CRED>
      Credential validated successfully

Back up a volume

  1. Enter the following pxctl volume list command to list all volumes on the source cluster:

      pxctl volume list
      ID            NAME    SIZE    HA  SHARED  ENCRYPTED   IO_PRIORITY STATUS              SNAP-ENABLED
  869510655149846346    testvol 1 GiB   1   no  no      HIGH        up - attached on 70.0.88.123    no
  186765995885697345    vol2    1 GiB   1   no  no      HIGH        up - attached on 70.0.88.123    no

  2. To back up a volume, enter the following pxctl cloudsnap backup command, specifying the name of your volume. The following example backs up a volume called testvol:

      pxctl cloudsnap backup testvol
      Cloudsnap backup started successfully with id: 0be453e1-ec7a-4db7-9724-a46868cc6b5c

  3. Enter the pxctl cloudsnap status command to display the status of your backup or restore operations:

      pxctl cloudsnap status
      NAME                  SOURCEVOLUME        STATE       NODE        TIME-ELAPSED    COMPLETED
  5c5d3afa-6579-465e-9e34-9bff6ea440eb  869510655149846346  Backup-Failed   70.0.87.153 80.915632ms Wed, 22 Jan 2020 23:51:17 UTC
  e44b3fb4-45f6-4a83-980b-10458b7a8445  869510655149846346  Backup-Done 70.0.87.153 55.098204ms Wed, 22 Jan 2020 23:52:15 UTC
  8a32dd41-931b-4ccf-8b99-f15839b26e76  186765995885697345  Backup-Failed   70.0.87.153 39.703754ms Wed, 29 Jan 2020 18:17:30 UTC
  7ddc9d23-541c-41d3-90c6-2f4a504c01f9  186765995885697345  Backup-Done 70.0.87.153 60.439873ms Wed, 29 Jan 2020 18:34:17 UTC
  0be453e1-ec7a-4db7-9724-a46868cc6b5c  869510655149846346  Backup-Done 70.0.87.153 45.874676ms Wed, 29 Jan 2020 22:32:30 UTC

  4. To see more details about your backup operation, enter the pxctl cloudsnap status command specifying the following:

    • The --json flag
    • The --name flag with the task name of your backup.

    Example:

      pxctl --json cloudnsap status --name 0be453e1-ec7a-4db7-9724-a46868cc6b5c
      0be453e1-ec7a-4db7-9724-a46868cc6b5c
  {
  "0be453e1-ec7a-4db7-9724-a46868cc6b5c": {
  "ID": "3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-1140911084048715440",
  "OpType": "Backup",
  "Status": "Done",
  "BytesDone": 368640,
  "BytesTotal": 0,
  "EtaSeconds": 0,
  "StartTime": "2020-01-29T22:32:30.258745865Z",
  "CompletedTime": "2020-01-29T22:32:30.304620541Z",
  "NodeID": "a5f87c11-05c5-41b4-84e5-3c38a8c04736",
  "SrcVolumeID": "869510655149846346",
  "Info": [
  ""
  ],
  "CredentialUUID": "0d9847d6-786f-4ed8-b263-5cde5a5a12f5",
  "GroupCloudBackupID": ""
  }

  5. Run the pxctl cloudsnap list command, and look through the output to find the identifier of the cloud snapshot associated with your volume. You will use this to restore your cloud snapshot.

      pxctl cloudsnap list
      SOURCEVOLUME      SOURCEVOLUMEID          CLOUD-SNAP-ID                                       CREATED-TIME                TYPE        STATUS
  testvol           869510655149846346      3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-457116323485794032      Wed, 22 Jan 2020 23:52:15 UTC       Manual      Done
  vol2          186765995885697345      3f2fa12e-186f-466d-ac35-92cf569c9358/186765995885697345-237744851553132030      Wed, 29 Jan 2020 18:34:17 UTC       Manual      Done
  testvol           869510655149846346      3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-1140911084048715440     Wed, 29 Jan 2020 22:32:30 UTC       Manual      Done

    The CLOUD-SNAP-ID column is in the form of <YOUR_SOURCE_CLUSTER_ID>/<YOUR_CLOUD_SNAP_ID>. In this example, the identifier of the source cluster is 3f2fa12e-186f-466d-ac35-92cf569c9358, and the identifier of the cloud snapshot is 869510655149846346-457116323485794032.

Create your cloud snapshot credentials on the destination cluster

  1. Enter the pxctl credentials create command, specifying the following:

    • The --provider flag with the name of the cloud provider (s3).
    • The --s3-access-key flag with your secret access key
    • The --s3-secret-key flag with your access key ID
    • The --s3-region flag with the name of the S3 region (us-east-1)
    • The --s3-endpoint flag with the name of the endpoint (s3.amazonaws.com)
    • The optional --s3-storage-class flag with either the STANDARD or STANDARD-IA value, depending on which storage class you prefer
    • The name of your cloud credentials

    Example:

      pxctl credentials create --provider s3 --s3-access-key <YOUR_ACCESS_KEY> --s3-secret-key <YOUR_SECRET_KEY> --s3-region us-east-1 --s3-endpoint <YOUR_ENDPOINT> --s3-storage-class <YOUR_STORAGE_CLASS> <YOUR_DEST_S3_CRED>
      Credentials created successfully, UUID:bb281a27-c2bb-4b3d-b5b9- efa0316a9561

Restore your volume on the target cluster

  1. On the target cluster, verify that your cloud snapshot is visible. Enter the pxctl cloudsnap list command, specifying the --cluster flag with the identifier of the source cluster.

    Example:

      pxctl cloudsnap list --cluster 3f2fa12e-186f-466d- ac35-92cf569c9358
      3f2fa12e-186f-466d-ac35-92cf569c9358
  SOURCEVOLUME      SOURCEVOLUMEID          CLOUD-SNAP-ID                                       CREATED-TIME                TYPE        STATUS
  testvol           869510655149846346      3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-457116323485794032      Wed, 22 Jan 2020 23:52:15 UTC       Manual      Done
  vol2          186765995885697345      3f2fa12e-186f-466d-ac35-92cf569c9358/186765995885697345-237744851553132030      Wed, 29 Jan 2020 18:34:17 UTC       Manual      Done
  testvol           869510655149846346      3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-1140911084048715440     Wed, 29 Jan 2020 22:32:30 UTC       Manual      Done

  2. To restore your volume, run the pxctl cloudsnap restore command specifying the --snap flag with the cloud snapshot identifier associated with your backup. Example:

      pxctl cloudsnap restore --snap 3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-1140911084048715440
      Cloudsnap restore started successfully on volume: 1127186980413628688 with task name:e306a2e0-4c88-426c-ae88-a6b731f73983

  3. To see the status of your restore operation, enter the following command:

      pxctl cloudsnap status
      NAME                                  SOURCEVOLUME                                    STATE       NODE        TIME-ELAPSED    COMPLETED
  2e53ca62-8289-498b-ad9d-dd77c14c00bc  79001397979145130                               Backup-Done 70.0.91.94  44.634974ms Wed, 29 Jan 2020 20:13:58 UTC
  6304dfb7-2f9f-4236-9392-2aba15c5b300  3f2fa12e-186f-466d-ac35-92cf569c9358/869510655149846346-1140911084048715440 Restore-Done    70.0.91.94  53.527074ms Wed, 29 Jan 2020 22:52:47 UTC

  4. Run the pxctl volume list command to list all volumes on the destination cluster:

      pxctl volume list
      ID            NAME                    SIZE    HA  SHARED  ENCRYPTED   IO_PRIORITY STATUS      SNAP-ENABLED
  1021141073379827532   Restore-869510655149846346-556794585    1 GiB   1   no  no      HIGH        up - detached   no
  79001397979145130 samvol                  1 GiB   1   no  no      HIGH        up - detached   no

The naming scheme for cloud backups

Cloud backups adhere to the following naming scheme: <bucket-id>/<vol-id>-<snap-id>.

Example:

  • 2e4d4b67-95d7-481e-aec5-14223ac55170/56706279008755778-725134927222077463

For incremental backups, Portworx adds the -incr suffix as follows: <bucket-id>/<vol-id>-<snap-id>-incr.

Example:

  • 2e4d4b67-95d7-481e-aec5-14223ac55170/590114184663672482-951325819047337066-incr
arrow_back_ios Previous
Cloud backups for single PVCs
Next arrow_forward_ios
Snapshot single PVCs
Last edited: Friday, Apr 7, 2023
Questions? Visit the Portworx forum.