snaps-cloud
This document shows how you can create cloud snapshots of Portworx volumes and how you can clone those snapshots to use them in pods.
You cannot use an older version of Portworx to restore a cloud snapshot created with a newer one. For example, if you're running Portworx 2.1, you can't restore a cloud snapshot created with Portworx 2.2.
Back up a volume and restore it to the same Portworx cluster
This section shows how you can back up a volume and restore it to the same Portworx cluster.
Prerequisites
Configuring cloud secrets
To create cloud snapshots, one needs to setup secrets with Portworx which will get used to connect and authenticate with the configured cloud provider. Follow instructions on the create and configure credentials section to setup secrets.
Create cloud snapshots
With cloud snapshots, you can either snapshot individual PVCs one by one or snapshot a group of PVCs.
Restore cloud snapshots
Once you've created a cloud snapshot, you can restore it to a new PVC or the original PVC.
Restore a cloud snapshot to a new PVC
When you install Stork, it also creates a storage class called stork-snapshot-sc. This storage class can be used to create PVCs from snapshots.
To create a PVC from a snapshot, add the snapshot.alpha.kubernetes.io/snapshot
annotation to refer to the snapshot name. If the snapshot exists in another namespace, you should specify the snapshot namespace with the stork.libopenstorage.org/snapshot-source-namespace
annotation in the PVC.
The Retain policy is important if you need to keep the volume in place, even after removing the Kubernetes objects from a cluster.
- As shown in the following example, the storageClassName should be the Stork StorageClass
stork-snapshot-sc
. - When using this storage class the PVC is creating with
delete
as Retain policy. However, if the source PVC is having the policy asretain
, then this will not be inherited to the restored PVC. After the restore, you should manually verify the retain policy and change it if needed.
-
Create a new PVC using the following spec:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: vdbench-restore
namespace: vdbench-sv4-svc-autojournal
annotations:
snapshot.alpha.kubernetes.io/snapshot: vdbench-pvc-output-sv4-svc-schedule-interval-2024-01-10-225924
stork.libopenstorage.org/snapshot-source-namespace: vdbench-sv4-svc-autojournal
spec:
accessModes:
- ReadWriteOnce
storageClassName: stork-snapshot-sc
resources:
requests:
storage: 10Gi -
Once the above PVC specification is applied, verify that Stork has created a PVC that is backed by a clone of the specified Portworx volume snapshot(
vdbench-pvc-output-sv4-svc-schedule-interval-2024-01-10-225924
):storkctl -n vdbench-sv4-svc-autojournal get volumesnapshot
NAME PVC STATUS CREATED COMPLETED TYPE
vdbench-pvc-output-sv4-svc-schedule-interval-2024-01-10-225924 vdbench-pvc-output-sv4-svc Ready 10 Jan 24 14:59 PST 10 Jan 24 14:59 PST cloud -
Verify that a cloud snapshot is restored to the PVC created above:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
vdbench-pvc-enc-sv4-svc Bound pvc-xxxxxxxx-xxxx-xxxx-xxxx-a2959220f31b 50Gi RWX vdbench-sc-sv4-svc-auto 29m
vdbench-pvc-output-sv4-svc Bound pvc-xxxxxxxx-xxxx-xxxx-xxxx-ceda98f2ae06 5Gi RWX vdbench-sc-sv4-svc-auto 29m
vdbench-restore Bound pvc-xxxxxxxx-xxxx-xxxx-xxxx-9916a6098418 10Gi RWO stork-snapshot-sc 4m46s
In the above example output, the vdbench-restore
PVC is in a Bound
status and is associated with the correct volume that represents the restored snapshot.
Restore a cloud snapshot to the original PVC
When you perform an in-place restore to a PVC, Stork takes the pods using that PVC offline, restores the volume from the snapshot, then brings the pods back online.
In-place restore using VolumeSnapshotRestore works only for applications deployed using the stork
scheduler.
If you're not using the Stork scheduler, Portworx displays the following error when describing the VolumeSnapshotRestore resource:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Failed 5s (x2 over 15s) stork application not scheduled by stork scheduler
-
Create a
VolumeSnapshotRestore
YAML file specifying the following:- apiVersion as
stork.libopenstorage.org/v1alpha1
- kind as
VolumeSnapshotRestore
- metadata.name with the name of the object that performs the restore
- metadata.namespace with the name of the target namespace
- spec.sourceName with the name of the snapshot you want to restore
- spec.sourceNamespace with the namespace in which the snapshot resides
The following example restores data from a snapshot called
mysql-snapshot
which was created in themysql-snap-restore-splocal
namespace to a PVC calledmysql-snap-inrestore
in thedefault
namespace:apiVersion: stork.libopenstorage.org/v1alpha1
kind: VolumeSnapshotRestore
metadata:
name: mysql-snap-inrestore
namespace: default
spec:
sourceName: mysql-snapshot
sourceNamespace: mysql-snap-restore-splocal - apiVersion as
-
Place the spec into a file called
mysql-cloud-snapshot-restore.yaml
and apply it:
-
You can enter the following command to see the status of the restore process:
storkctl get volumesnapshotrestore
NAME SOURCE-SNAPSHOT SOURCE-SNAPSHOT-NAMESPACE STATUS VOLUMES CREATED
mysql-snap-inrestore mysql-snapshot default Successful 1 23 Sep 19 21:55 EDT
Name: mysql-snap-inrestore
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"stork.libopenstorage.org/v1alpha1","kind":"VolumeSnapshotRestore","metadata":{"annotations":{},"name":"mysql-snap-inrestore...
API Version: stork.libopenstorage.org/v1alpha1
Kind: VolumeSnapshotRestore
Metadata:
Creation Timestamp: 2019-09-23T17:24:30Z
Generation: 5
Resource Version: 904014
Self Link: /apis/stork.libopenstorage.org/v1alpha1/namespaces/default/volumesnapshotrestores/mysql-snap-inrestore
UID: xxxxxxxx-xxxx-xxxx-xxxx-000c295d6364
Spec:
Group Snapshot: false
Source Name: mysql-snapshot
Source Namespace: default
Status:
Status: Successful
Volumes:
Namespace: default
Pvc: mysql-data
Reason: Restore is successful
Snapshot: k8s-volume-snapshot-xxxxxxxx-xxxx-xxxx-xxxx-320ff611f4ca
Status: Successful
Volume: pvc-xxxxxxxx-xxxx-xxxx-xxxx-000c295d6364
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Successful 0s stork Snapshot in-Place Restore completed
References
Back up a volume and restore it to a different Portworx cluster
This section shows how you can back up a volume and restore it to a different Portworx cluster using the pxctl
command-line utility.
Prerequisites
Before you can back up and restore a volume to a different Portworx cluster, you must meet the following prerequisites:
Create your cloud snapshot credentials on the source cluster
The options you use to create your cloud snapshot credentials differ based on which secret store provider you use. The steps in this document describe AWS KMS, but you can find instructions for creating other credentials in the CLI reference.
-
Enter the
pxctl credentials create
command, specifying the following:- The
--provider
flag with the name of the cloud provider (s3
). - The
--s3-access-key
flag with your secret access key - The
--s3-secret-key
flag with your access key ID - The
--s3-region
flag with the name of the S3 region (us-east-1
) - The
--s3-endpoint
flag with the name of the endpoint (s3.amazonaws.com
) - The optional
--s3-storage-class
flag with either theSTANDARD
orSTANDARD-IA
value, depending on which storage class you prefer - The name of your cloud credentials
Example:
pxctl credentials create --provider s3 --s3-access-key <YOUR_ACCESS_KEY> --s3-secret-key <YOUR_SECRET_KEY> --s3-region us-east-1 --s3-endpoint <YOUR_ENDPOINT> --s3-storage-class <YOUR_STORAGE_CLASS> <YOUR_SOURCE_S3_CRED>
Credentials created successfully, UUIDU0d9847d6-786f-4ed8- b263-5cde5a5a12f5
- The
-
You can validate your cloud snapshot credentials by entering the
pxctl credentials validate
command followed by the name of your cloud credentials:pxctl cred validate <YOUR_SOURCE_S3_CRED>
Credential validated successfully
Back up a volume
-
Enter the following
pxctl volume list
command to list all volumes on the source cluster:pxctl volume list
ID NAME SIZE HA SHARED ENCRYPTED IO_PRIORITY STATUS SNAP-ENABLED
869510655149846346 testvol 1 GiB 1 no no HIGH up - attached on X.X.X.123 no
186765995885697345 vol2 1 GiB 1 no no HIGH up - attached on X.X.X.123 no -
To back up a volume, enter the following
pxctl cloudsnap backup
command, specifying the name of your volume. The following example backs up a volume calledtestvol
:pxctl cloudsnap backup testvol
Cloudsnap backup started successfully with id: xxxxxxxx-xxxx-xxxx-xxxx-a46868cc6b5c
-
Enter the
pxctl cloudsnap status
command to display the status of your backup or restore operations:pxctl cloudsnap status
NAME SOURCEVOLUME STATE NODE TIME-ELAPSED COMPLETED
xxxxxxxx-xxxx-xxxx-xxxx-9bff6ea440eb 869510655149846346 Backup-Failed X.X.X.153 80.915632ms Wed, 22 Jan 2020 23:51:17 UTC
xxxxxxxx-xxxx-xxxx-xxxx-10458b7a8445 869510655149846346 Backup-Done X.X.X.153 55.098204ms Wed, 22 Jan 2020 23:52:15 UTC
xxxxxxxx-xxxx-xxxx-xxxx-f15839b26e76 186765995885697345 Backup-Failed X.X.X.153 39.703754ms Wed, 29 Jan 2020 18:17:30 UTC
xxxxxxxx-xxxx-xxxx-xxxx-2f4a504c01f9 186765995885697345 Backup-Done X.X.X.153 60.439873ms Wed, 29 Jan 2020 18:34:17 UTC
xxxxxxxx-xxxx-xxxx-xxxx-a46868cc6b5c 869510655149846346 Backup-Done X.X.X.153 45.874676ms Wed, 29 Jan 2020 22:32:30 UTC -
To see more details about your backup operation, enter the
pxctl cloudsnap status
command specifying the following:-
The
--json
flag -
The
--name
flag with the task name of your backup.Example:
pxctl --json cloudnsap status --name xxxxxxxx-xxxx-xxxx-xxxx-a46868cc6b5c
xxxxxxxx-xxxx-xxxx-xxxx-a46868cc6b5c
{
"xxxxxxxx-xxxx-xxxx-xxxx-a46868cc6b5c": {
"ID": "xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-1140911084048715440",
"OpType": "Backup",
"Status": "Done",
"BytesDone": 368640,
"BytesTotal": 0,
"EtaSeconds": 0,
"StartTime": "2020-01-29T22:32:30.258745865Z",
"CompletedTime": "2020-01-29T22:32:30.304620541Z",
"NodeID": "xxxxxxxx-xxxx-xxxx-xxxx-3c38a8c04736",
"SrcVolumeID": "869510655149846346",
"Info": [
""
],
"CredentialUUID": "xxxxxxxx-xxxx-xxxx-xxxx-5cde5a5a12f5",
"GroupCloudBackupID": ""
}
-
-
Run the
pxctl cloudsnap list
command, and look through the output to find the identifier of the cloud snapshot associated with your volume. You will use this to restore your cloud snapshot.pxctl cloudsnap list
SOURCEVOLUME SOURCEVOLUMEID CLOUD-SNAP-ID CREATED-TIME TYPE STATUS
testvol 869510655149846346 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-457116323485794032 Wed, 22 Jan 2020 23:52:15 UTC Manual Done
vol2 186765995885697345 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/186765995885697345-237744851553132030 Wed, 29 Jan 2020 18:34:17 UTC Manual Done
testvol 869510655149846346 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-1140911084048715440 Wed, 29 Jan 2020 22:32:30 UTC Manual DoneThe
CLOUD-SNAP-ID
column is in the form of<YOUR_SOURCE_CLUSTER_ID>/<YOUR_CLOUD_SNAP_ID>
. In this example, the identifier of the source cluster isxxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358
, and the identifier of the cloud snapshot is869510655149846346-457116323485794032
.
Create your cloud snapshot credentials on the destination cluster
-
Enter the
pxctl credentials create
command, specifying the following:- The
--provider
flag with the name of the cloud provider (s3
). - The
--s3-access-key
flag with your secret access key - The
--s3-secret-key
flag with your access key ID - The
--s3-region
flag with the name of the S3 region (us-east-1
) - The
--s3-endpoint
flag with the name of the endpoint (s3.amazonaws.com
) - The optional
--s3-storage-class
flag with either theSTANDARD
orSTANDARD-IA
value, depending on which storage class you prefer - The name of your cloud credentials
Example:
pxctl credentials create --provider s3 --s3-access-key <YOUR_ACCESS_KEY> --s3-secret-key <YOUR_SECRET_KEY> --s3-region us-east-1 --s3-endpoint <YOUR_ENDPOINT> --s3-storage-class <YOUR_STORAGE_CLASS> <YOUR_DEST_S3_CRED>
Credentials created successfully, UUID:bb281a27-c2bb-4b3d-b5b9- efa0316a9561
- The
Restore your volume on the target cluster
-
On the target cluster, verify that your cloud snapshot is visible. Enter the
pxctl cloudsnap list
command, specifying the--cluster
flag with the identifier of the source cluster.Example:
pxctl cloudsnap list --cluster 3f2fa12e-186f-466d- ac35-92cf569c9358
xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358
SOURCEVOLUME SOURCEVOLUMEID CLOUD-SNAP-ID CREATED-TIME TYPE STATUS
testvol 869510655149846346 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-457116323485794032 Wed, 22 Jan 2020 23:52:15 UTC Manual Done
vol2 186765995885697345 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/186765995885697345-237744851553132030 Wed, 29 Jan 2020 18:34:17 UTC Manual Done
testvol 869510655149846346 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-1140911084048715440 Wed, 29 Jan 2020 22:32:30 UTC Manual Done -
To restore your volume, run the
pxctl cloudsnap restore
command specifying the--snap
flag with the cloud snapshot identifier associated with your backup. Example:pxctl cloudsnap restore --snap xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-1140911084048715440
Cloudsnap restore started successfully on volume: 1127186980413628688 with task name:xxxxxxxx-xxxx-xxxx-xxxx-a6b731f73983
-
To see the status of your restore operation, enter the following command:
pxctl cloudsnap status
NAME SOURCEVOLUME STATE NODE TIME-ELAPSED COMPLETED
xxxxxxxx-xxxx-xxxx-xxxx-dd77c14c00bc 79001397979145130 Backup-Done X.X.X.94 44.634974ms Wed, 29 Jan 2020 20:13:58 UTC
xxxxxxxx-xxxx-xxxx-xxxx-2aba15c5b300 xxxxxxxx-xxxx-xxxx-xxxx-92cf569c9358/869510655149846346-1140911084048715440 Restore-Done X.X.X.94 53.527074ms Wed, 29 Jan 2020 22:52:47 UTC -
Run the
pxctl volume list
command to list all volumes on the destination cluster:pxctl volume list
ID NAME SIZE HA SHARED ENCRYPTED IO_PRIORITY STATUS SNAP-ENABLED
1021141073379827532 Restore-869510655149846346-556794585 1 GiB 1 no no HIGH up - detached no
79001397979145130 samvol 1 GiB 1 no no HIGH up - detached no
The naming scheme for cloud backups
Cloud backups adhere to the following naming scheme: <bucket-id>/<vol-id>-<snap-id>
.
Example:
xxxxxxxx-xxxx-xxxx-xxxx-14223ac55170/56706279008755778-725134927222077463
For incremental backups, Portworx adds the -incr
suffix as follows: <bucket-id>/<vol-id>-<snap-id>-incr
.
Example:
xxxxxxxx-xxxx-xxxx-xxxx-14223ac55170/590114184663672482-951325819047337066-incr