Skip to main content
Version: 3.0

Upgrade Portworx Backup

Upgrade workflow​

  1. On Portworx Backup cluster:

  2. On application cluster, Upgrade Stork.

    After upgrading, see the Configure and Working with Portworx Backup modules for next steps.

Prerequisites​

Before starting an upgrade, ensure the following conditions are met:

  • Check component compatibility: Verify your Stork and Portworx Enterprise versions are compatible with the target Portworx Backup version. See the Compatibility Matrix.
  • kubectl access: Your kubectl context is set to the Portworx Backup cluster with admin privileges.
  • Helm version: Helm 3 is installed on the machine running the upgrade command.
  • MongoDB pod health: All three MongoDB pods (mongo-0, mongo-1, mongo-2) are in Ready state. The upgrade will fail if any MongoDB pod is not ready. See Pre-upgrade job for recovery steps.
  • PVC storage: Sufficient storage is available on the PVCs used by Portworx Backup components.
  • pxc-credentials secret: The pxc-credentials secret is present in the Portworx Backup namespace with valid database passwords. See Password Policy.
  • Backup your data: Take a backup of critical data before upgrading.
  • Version compatibility: Verify the target version is compatible with your current Stork and Portworx Enterprise versions. See the Compatibility Matrix.

Portworx Backup supports upgrades to version n from versions n-1 or n-2. For example, if you are on Portworx Backup version 2.10.x or 2.11.x, you can directly upgrade to 3.0.0. If you are on 2.9.x or earlier and want to upgrade to 3.0.0, you need to upgrade in a phased manner. First upgrade to an intermediate version (2.10.x or 2.11.x), and then upgrade to 3.0.0.

Upgrading to Portworx Backup 2.10.0 or above now requires the pxc-credentials secret for password management. For more information, see the Password Policy guide.

Upgrade matrix​

Portworx Backup VersionIntermediate Upgrade
Required?		Upgrade Path
2.11.xNo2.11.x → 3.0.x
2.10.xNo2.10.x → 3.0.x
2.9.xYes2.9.x → 2.11.x → 3.0.x
2.8.xYes2.8.x → 2.10.x → 3.0.x
2.7.xYes2.7.x → 2.9.x → 2.11.x → 3.0.x
2.6.xYes2.6.x → 2.8.x → 2.10.x → 3.0.x
2.5.xYes2.5.x → 2.7.x → 2.9.x → 2.11.x → 3.0.x
2.4.xYes2.4.x → 2.6.x → 2.8.x → 2.10.x → 3.0.x
2.3.xYes2.3.x → 2.5.x → 2.7.x → 2.9.x → 2.11.x → 3.0.x
2.2.xYes2.2.x → 2.4.x → 2.6.x → 2.8.x → 2.10.x → 3.0.x
2.1.xYes2.1.x → 2.3.x → 2.5.x → 2.7.x → 2.9.x → 2.11.x → 3.0.x
caution

If you encounter an upgrade failure while upgrading to the latest version of Portworx Backup, avoid rolling back to your previous version using Helm rollback or any other methods. In such situations, do not revert to the previously installed version. Instead, reach out to the Portworx Backup Support team for assistance.

Pre-upgrade job​

Starting from Portworx Backup version 2.7.3, a pre-upgrade job checks the health of the pods in the MongoDB StatefulSet to ensure that they are in Ready state before proceeding with the upgrade. The upgrade fails if all three MongoDB pods in the StatefulSet are not in the Ready state. By default, Portworx Backup assigns mongo-0 as the primary pod and mongo-1 and mongo-2 become secondary pods.

Before you proceed with the Helm upgrade to 2.7.3 or above from previous versions of Portworx Backup, ensure that all MongoDB pods are in Running state. If they are not, two cases can arise:

Case 1​

Secondary pod is not in Ready state

If a secondary pod (mongo-1 or mongo-2) is not in Ready state:

  • Identify the secondary failing pod that is not in the Ready state.

  • Delete the PVC (Persistent Volume Claim) associated with the failing pod.

  • Delete the pod to allow it to be recreated.

Ensure that all pods have reached Ready state and continue with the Helm upgrade.

Case 2​

Primary pod is not in Ready state

Due to issues in the node, network, or underlying storage, if mongo-0 becomes unresponsive for some time, one of the secondary pods (mongo-1 or mongo-2) gets stepped up as the primary pod. If mongo-0 recovers but is still not in Ready state, perform the following steps to bring the mongo-0 pod to Ready state:

  1. Sync the data from the primary (mongo-1 or mongo-2) to the mongo-0 pod by executing the following steps:

    a. Run the following command to enter MongoDB shell from mongo-0 pod:

    mongosh admin -u root -p pxcentral

    b. Run the following commands to find out the primary pod:

    rs.status()

    Output string:

      members: [
        {
          _id: 1,
          name: 'pxc-backup-mongodb-1.pxc-backup-mongodb-headless:<port-number>',
          health: 1,
          state: 1,
          stateStr: 'PRIMARY',

    c. Run the following command to set the highest priority to the primary pod (that you have got from the output of step b):

      cfg = rs.conf()
      cfg.members[0].priority = 1
      cfg.members[1].priority = 5
      cfg.members[2].priority = 1
      rs.reconfig(cfg)
      rs.status()

Wait until the sync process completes.

To make mongo-0 primary pod again:

  1. Enter mongo shell from mongo-1 with the following command:

    mongosh admin -u root -p pxcentral

  2. Now run the following commands to set mongo-0 as primary pod:

    cfg = rs.conf()
    cfg.members[0].priority = 5
    cfg.members[1].priority = 1
    cfg.members[2].priority = 1
    rs.reconfig(cfg)
    rs.status()

    Once all the pods are in the Ready state, proceed with the Helm upgrade.

    If the primary or any secondary pod cannot be recovered with the steps above, contact the Portworx Backup support team.

KDMP backup behavior for upgrades​

Assume that you are on Portworx Backup version 2.6.0 or below, with cloud native driver installed on your application clusters. When you upgrade to Portworx Backup version 2.7.0 and above from previous versions, the existing manual backups will not be affected:

The backups that are scheduled (to be created in the future) with the cloud native driver:

  • Get migrated to CSI snapshots along with the offload-to-backup-location option if a volume snapshot class is available

  • Use the KDMP driver to create a direct KDMP backup if a volume snapshot class is not available

    note

    Only scheduled backups created using the cloud native driver before the upgrade are migrated; backups created after the upgrade are not. Scheduled backups created with other drivers remain unchanged. One of the backups might fail after migrating from the native driver to CSI + Offload if the reconciler has not yet updated the schedule CR.

    note

    Helm upgrade fails if all the three mongoDB pods are not in Ready state. You can upgrade to Portworx Backup version 2.7.3 or above from previous versions successfully only if all the three mongoDB pods are in Ready state. Before the Helm upgrade:

    • Delete the pod and the PVC associated with the failing pod.
    • Scale up the replica set to three and ensure that all the pods are in Ready state.

Following sections guide you to upgrade your Portworx Backup (in non-airgapped and air-gapped environments) and Stork component:

Last updated on