Skip to main content
Version: 3.2

Upgrade Kubernetes with Portworx on GCP Anthos

This document explains how to leverage the smart upgrade feature of Portworx to upgrade multiple Kubernetes nodes in parallel without any disruption.

Smart upgrade​

The Smart upgrade feature introduces a streamlined, resilient upgrade process for Kubernetes nodes, allowing them to be upgraded in parallel while maintaining volume quorum and without application disruption.

The upgrade process for Kubernetes clusters is streamlined using per-node PodDisruptionBudgets (PDBs). Operator creates a dedicated PDB for each Portworx storage node. These per-node PDBs ensure granular control over node disruptions, allowing parallel upgrades without risking volume quorum.

During upgrades, the Operator dynamically adjusts the PDBs to enable safe draining of nodes selected for upgrade. Nodes are carefully chosen to avoid disrupting volume availability, with volume provisioning disabled on upgrading nodes. This method significantly reduces upgrade times, enhances cluster resilience, and maintains high availability throughout the process.

By default, smart upgrade is disabled, and cluster-wide PDB will be used where minAvailable set to numStorageNodes - 1, which means one Kubernetes node is upgraded at a time.

You can enable the smart upgrade by setting the portworx.io/disable-non-disruptive-upgrade annotation to false. Also, you can configure the minimum number of nodes that must be available at a time using the portworx.io/storage-pdb-min-available annotation in the StorageCluster.

The following are the key benefits of using smart upgrades:

  • Parallel upgrades: Based on volume distribution, the Portworx Operator tries the best to select multiple nodes for concurrent upgrades, accelerating the upgrade process while eliminating downtime and application disruption.
  • Volume quorum maintenance: Ensures volume quorum is maintained throughout the upgrade process.
  • Managed node upgrades: You can use the portworx.io/storage-pdb-min-available annotation in the StorageCluster CRD to manage the number of nodes upgraded in parallel.
  • Automatic reconciliation: The Portworx operator actively monitors and reconciles the storage nodes during upgrades, ensuring smooth progression while preserving quorum integrity.
important
  • There will be a downtime for applications using volumes with a replication factor of 1.
  • Smart upgrade is not supported for synchronous DR setup.

Prerequisites​

For smart upgrades, ensure the following prerequisites are met:

  • Required Portworx and Operator versions:
    • Portworx version 3.1.2 or later
    • Operator version 24.2.0 or later
  • The cluster must be ready and available for upgrade. You can use the pxctl status and kubectl get storagenodes -n portworx commands to check the cluster status.
    • No nodes or pools should be under maintenance.
    • No decommissioned nodes should appear in the output of the kubectl get storagenodes command.
    • No nodes should have the px/service=stop or px/service=disabled label. If nodes have these labels, remove them and restart the Portworx service or decommission the node before the upgrade.

Upgrade Kubernetes​

  1. To upgrade your Kubernetes cluster using the smart upgrade feature, make the following changes in the StorageCluster:

    1. Enable the smart upgrade by setting the portworx.io/disable-non-disruptive-upgrade annotation to false.

      note

      When smart upgrade is enabled, the operator uses quorum+1 as the minAvailable value by default. If you want to override the value, follow the next step.

    2. (Optional) Set the minimum number of nodes that must be available during upgrade using the portworx.io/storage-pdb-min-available annotation.

    apiVersion: core.libopenstorage.org/v1
    kind: StorageCluster
    metadata:
      name: portworx
      namespace: <px-namespace>
      annotations:
        portworx.io/disable-non-disruptive-upgrade: "false"
        # If you want to override the default value of `minAvailable`, uncomment the below line and set a desired value.
        # portworx.io/storage-pdb-min-available: "2"

  2. Upgrade your Kubernetes cluster.

note

When storage-pdb-min-availablevalue is set in StorageCluster during upgrade from version 1.29 to 1.30, you must invoke the user cluster upgrade command with --skip-validation-all and --skip-pre-upgrade-checks=true flags.

Last updated on