KDMP backups
KDMP backups are generic backups that Portworx Backup creates utilizing the KDMP driver. If you prefer to use Portworx Backup's KDMP driver, you can enable Cross cloud Backup/restore option during creation of backup.
KDMP backup types
Portworx backup supports two variants of KDMP backups as outlined in the following sections:
- Direct KDMP/Generic
In this backup type, Portworx Backup utilizes the KDMP driver to attach the mount point of the PV on the worker node and copies the file system contents to object storage.
When a direct KDMP backup or generic backup occurs
Portworx Backup triggers a direct KDMP backup in the following scenarios:
- When you enable the Cross Cloud Backup/Restore option in the Create Backup window of the web console during back creation operation.
- KDMP with local snapshot
In Portworx Backup, CSI driver pitches in to take local snapshots and KDMP driver later offloads these snapshots to S3 or NFS backup location.
In this case Portworx Backup performs the following tasks:
a. Takes available snapshot storageclass information
b. Creates a snapshot using CSI driver
c. Uses KDMP driver to attach the mount point of the PV on the worker node
d. Copies the file system contents to S3 or NFS backup location
When a KDMP backup occurs
- If you have an environment with Portworx Backup and a different storage management platform other than Portworx Enterprise:
-
If the storage provider does not support CSI snapshots
- When storage provider does not support snapshot mechanism with CSI driver. For some on-premises (for example, Flash Array) or on cloud storage providers without CSI snapshot support, direct KDMP backup gets triggered and you need to choose Cross Cloud Backup/Restore option in Create Backup window during backup creation operation.
-
If the storage provider supports CSI snapshots and if you want to offload the backup to a backup location (S3 or NFS) along with the selection of volume snapshot class during creation of backup.
KDMP backup support matrix
Following table provides KDMP backup support matrix when the storage provider supports CSI snapshot:
| Cloud storage provider | Storage type | Backup mechanism |
|---|---|---|
| AWS | EBS |
|
| EFS | Direct KDMP | |
| Azure | Disk |
|
| File share | CSI driver (if you select Offload to backup location with CSI snapshot class option in the user interface) | |
| IBM | IBM storage |
|
| GCP | Google disk |
|
| Google file | Direct KDMP | |
| OCP | OCP storage |
|
-
Cross-cloud and cross-region backups and restores with Portworx Backup.
KDMP-config ConfigMap
In some scenarios, you may have to force a KDMP backup by updating the kdmp-config` ConfigMap as outlined in the section below:
Exclude files and directories
Consider a scenario where you have directories or files (hidden, read-only, logs) with different extensions and you wish to exclude those from being included in the backup. Portworx Backup now allows you to exclude such directories or files from being backed up. To exclude or omit such directories and files from being included in the backup, you need to update the kdmp-config ConfigMap with the following steps:
- Run the following command to edit the
kdmp-configConfigMap:
kubectl edit cm kdmp-config -n kube-system
- (Optional, execute the following command only if you want direct KDMP backup in cases where it is native CSI backup)
Add the following parameter in the data section:
BACKUP_TYPE: "Generic"
You must add the above parameter on the source application cluster where you perform backup. It is not mandatory to add the parameter on the destination cluster where you restore.
- Add the following key-value pair:
KDMP_EXCLUDE_FILE_LIST: |
<storageclassName1>=<dir-list>,<file-list1>,....
<storageclassName2>=<dir-list>,<file-list1>,....
Example
apiVersion: v1
data:
ENABLE_PX_GENERIC_BACKUP: "true"
KDMP_BACKUP_JOB_LIMIT: "5"
KDMP_DELETE_JOB_LIMIT: "5"
KDMP_EXCLUDE_FILE_LIST: |
px-db=dir1,file1,dir2
mysql=dir1,file1,dir2
KDMP_KOPIAEXECUTOR_IMAGE: kopiaexecutor:1.2.10
KDMP_KOPIAEXECUTOR_IMAGE_SECRET: ""
KDMP_KOPIAEXECUTOR_LIMIT_CPU: "0.2"
KDMP_KOPIAEXECUTOR_LIMIT_MEMORY: 1Gi
KDMP_KOPIAEXECUTOR_REQUEST_CPU: "0.1"
KDMP_KOPIAEXECUTOR_REQUEST_MEMORY: 700Mi
KDMP_MAINTENANCE_JOB_LIMIT: "5"
KDMP_RESTORE_JOB_LIMIT: "5"
SNAPSHOT_TIMEOUT: ""
kind: ConfigMap
metadata:
name: kdmp-config
namespace: kube-system
resourceVersion: "7120917"
uid: bd1e505b-0191-414c-aedb-0b1d413d4002
You can exclude directories and files from being backed up for both direct KDMP and KDMP with CSI (local snapshot) backup types. For the latter, you cannot exclude any directories or files from being backed up if the local snapshot already exists even after kdmp-config ConfigMap is updated with the key-value pair specified above. If you want to restore with excluded directories or files, delete the existing local snapshot and then restore the backup.
KDMP backups and restores with large volume PVCs
KDMP job pods consume increased amounts of memory for the large volume PVC backup and restore operations to backup locations. As a result, you may see out of memory alerts or a failure of the kdmp job pods that run on each application or target cluster. During these scenarios, Portworx by PureStorage recommends to increase the CPU and memory limit related parameters in the kdmp-config ConfigMap which resides in the kube-system namespace on the target cluster or application cluster.
Update the following parameters in kdmp-config ConfigMap to resolve OOM errors:
KDMP_KOPIAEXECUTOR_LIMIT_CPU: "0.4"
KDMP_KOPIAEXECUTOR_REQUEST_CPU: "0.2"
KDMP_KOPIAEXECUTOR_REQUEST_MEMORY: 2Gi
KDMP_KOPIAEXECUTOR_LIMIT_MEMORY: 4Gi
Portworx by PureStorage does not recommend initiating a KDMP backup in your KubeVirt VM environment on SeLinux enabled clusters (includes OpenShift Virtualization environments). If there is a need to initiate a KDMP backup for Portworx backed volumes ensure that you have a CSI provisioner with the volume mode configured to ReadWriteMany (RWX). For all other scenarios tabulated above, refrain from taking a KDMP backup.
Related topics: