Skip to main content
Version: 2.11

Health Check for Install and Upgrade

Introduction​

The Portworx Backup 2.8.3 release introduces the health check feature to streamline installation and upgrades while reducing failures and improving user experience. This check is designed to proactively validate environments and detect potential issues to help users in troubleshooting the issues that may appear during Portworx Backup install or upgrade. Health check is a job that runs before install or upgrade and aims to:

  • Streamline the installation and upgrade experience
  • Reduce failure rates through early detection and fail-fast mechanisms
  • Simplify debugging with clear insights and recommendations

The health check feature is available for all environments that are currently supported by Portworx Backup. Also, health check automatically deletes post-install hook job and eliminates the process of manual deletion during upgrade.

note

Health check validations are enabled by default during Portworx Backup installation or upgrade. In case you want to disable this check for testing purposes, you can do so by appending the following string in the Portworx Central specgen during install or upgrade:

--set pxbackup.skipValidations=true

This skips the health check validations during install or upgrade.

Key validations​

1. Kubernetes version validation​

  • Ensures compatibility by checking Kubernetes cluster versions before installation or upgrades.

  • Provides warnings if the cluster is running an unsupported version.

2. Namespace quota check for PVCs​

  • Validates the resource quota for Persistent Volume Claims (PVCs) of the namespace where Portworx Backup will be deployed or upgraded.

  • Prevents deployment or upgrade if PVC storage requirements are not met.

3. StorageClass validation​

  • Verifies the existence of the specified storage class.

  • Blocks the install/upgrade until the user provides a valid storage class.

4. Persistent Volume Claim cleanup validation​

  • Checks for leftover PVCs from previous installations.

  • Prevents installation/upgrade until stale resources (PVCs) are cleaned up.

    note

    Stale PVCs are Persistent Volume Claims left behind from a previous incomplete or failed installation. They may contain backup metadata and can cause data conflicts or loss during reinstallation. Remove them before retrying the installation.

5. Port information display in Helm Notes​

  • Displays port-related details directly in Helm notes for user convenience.

  • Includes links to documentation for detailed network prerequisites.

6. Pre-Upgrade Health Check​

  • Ensures all Portworx Backup pods are in a healthy state before upgrades.

  • Blocks upgrade if pods are not in a healthy state.

caution

If the primary MongoDB pod goes down at any time either before or after the Portworx Backup installation or upgrade, it is essential for the secondary peers to be healthy and DNS-resolvable to maintain a stable replica set.

7. Pre-install/pre-upgrade reports​

  • Collects and stores all warnings generated during installation or upgrades in the respective pod logs.

  • Provides the following easy-to-access command to retrieve the report with warnings:

    kubectl get cm px-central-report -n <pxb-namespace> -o go-template='{{ index .data "report-<release_version>" }}'

    Please note that <release_version> here is always 2.8.3 and above.

  • Generates a comprehensive pre-install and pre-upgrade report.

8. Unsupported Path Upgrades​

  • Blocks upgrades if the installed version is more than two versions (n-2) behind the target release (current release).

9. Validation for File-system-based provisioners​

  • Prevents installation on storage classes using file-system-based provisioners.

10. Image Validation​

  • Validates image repository and tags during installation and upgrade.

Pod Validation​

Portworx Backup validates the following pods for their health during upgrade:

StatefulSets:

  • alertmanager-px-backup-alertmanager (only for px-backup version greater than 2.7.0)
  • prometheus-px-backup-dashboard-prometheus (only for px-backup version greater than 2.7.0)
  • pxc-backup-mongodb
  • pxcentral-keycloak
  • pxcentral-keycloak-postgresql
  • pxcentral-mysql

Deployments:

  • px-backup
  • pxcentral-apiserver
  • pxcentral-backend
  • pxcentral-frontend
  • pxcentral-lh-middleware
  • prometheus-operator (only for px-backup version greater than 2.7.0)

List of unsupported provisioners​

Portworx Backup does not support the following cloud and on-premises file system-based provisioners:

Cloud Storage Provisioners

  • efs.csi.aws.com – Amazon EFS (Elastic File System) for AWS
  • kubernetes.io/azure-file – Azure Files (legacy provisioner)
  • file.csi.azure.com – Azure Files CSI driver
  • vpc.file.csi.ibm.io – IBM Cloud VPC File Storage
  • filestore.csi.storage.gke.io – Google Cloud Filestore

On-Prem Storage Provisioners

  • nfs-client – Generic NFS provisioner (can be used for on-premises NFS servers)

  • kubernetes.io/glusterfs – GlusterFS (typically used for on-premises storage)

Insecure registry​

By default, the health check job uses secure connections (TLS) to validate container images. However, in certain cases, you may need to allow image validation to proceed even when the registry uses self-signed or untrusted or invalid certificates. You need to set the insecureRegistry flag to true in the values.yaml file during Helm installation to achieve this.

When to set insecureRegistry to true?

You should enable insecureRegistry in the following scenarios:

  • Your container registry uses self-signed certificates.
  • You are working with a private, on-premise registry that does not have a valid TLS certificate.
  • You are running in an air-gapped environment where external certificate validation is not possible.
  • You are testing in a development or staging environment that does not enforce strict TLS security.

How to enable insecure registry mode?

To allow connections to an insecure registry, update the values.yaml file:

insecureRegistry: true

Alternatively, you can enable it during Helm installation using the --set flag in the Portworx Central specgen page:

--set images.insecureRegistry=true

Security considerations

Setting insecureRegistry to true allows connections to registries without verifying TLS certificates. This setting should not be used in production environments unless absolutely necessary. If your registry supports proper TLS certificates, it is recommended to keep insecureRegistry set to false for enhanced security. By default, insecureRegistry is set to false, to ensure secure container image validations.

Next Steps​

Last updated on