Install Portworx Backup in Federated Mode

Applicable to Federated mode only

Overview

Portworx Backup supports a Federated mode (also referred to as Managed Service Provider mode or Workload Identity mode) designed for large-scale, multi-cluster, and service provider environments.

In this mode:

• The Portworx Backup server is deployed on a dedicated Kubernetes cluster (the backup cluster) and sends instructions to application clusters.
• All backup operations are handled locally by Stork on each shoot cluster (application cluster).
• Cloud credentials are not stored centrally. Each shoot cluster connects directly to the backup location using workload identity mechanisms (for example, Azure Managed Identity).

Before installing Portworx Backup in Federated mode, review the Federated Mode Specifications to ensure your environment meets all requirements.

For information on supported operation modes for Portworx Backup, see Operation Modes.

Classic mode vs Federated mode: installation differences

Parameter	Classic mode	Federated mode
Installation Helm flag	Not applicable	Required; set via the Portworx Central Spec Generator
Stork version	23.x.x or later	26.3.0 or later
Installed on	Any Kubernetes cluster. For supported platforms, see Cluster prerequisites.	Any Kubernetes cluster (Kubernetes 1.30.x or later) as the dedicated backup cluster. For supported platforms, see Cluster prerequisites. For application cluster requirements, see Supported Kubernetes platforms.

For information on other differences between Classic mode and Federated mode, see Operation modes feature comparison.

Prerequisites

Before you begin, review the Deployment requirements to ensure your environment meets all software, permission, and access requirements for Federated mode.

Ensure that the following prerequisites are met on both the backup cluster and all application clusters:

• Backup cluster (where Portworx Backup is installed):

  • Kubernetes 1.30.x or later
  • Helm 3.x installed on the workstation used for installation
  • Network access to the Kubernetes API server of each shoot cluster (to push CRs)
  • Non-air-gapped environments: access to the Portworx Backup Helm chart repository (http://charts.portworx.io/) and the Portworx image registry.
  • Air-gapped environments: a private container registry accessible by the backup cluster nodes, pre-loaded with all required Portworx Backup images. For instructions on pulling and pushing images to your private registry, see Set up the installation environment.

• Application clusters (shoot or non-Gardener clusters):

  • Portworx Enterprise version 3.6.1 or later installed.

  • Stork 26.3.0 or later installed on each application cluster (shoot or non-Gardener), deployed through the Portworx Operator 26.2.0 or later as part of a Portworx Enterprise installation. Standalone Stork deployments are not supported in Federated mode.

  • Azure Managed Identity provisioned and bound to the Stork service account

    note

    Only Azure Blob Storage is supported as a backup location in Federated mode in this release.

  • The Managed Identity must have the required Azure roles on the target Azure Blob Storage account. The required roles depend on whether immutable blob storage is enabled:

    • Standard (non-immutable) Azure Blob Storage: Assign the Storage Blob Data Contributor role.

    • Immutable Azure Blob Storage: The Storage Blob Data Contributor role alone is not sufficient for immutable storage accounts. In addition to the base Portworx permissions (portworx-wli-role), you must create and assign a custom role (for example, portworx-backup-locked-role) with the following permissions:

      • Storage account permissions

        • Microsoft.Storage/storageAccounts/read

      • Container permissions

        • Microsoft.Storage/storageAccounts/blobServices/containers/read
        • Microsoft.Storage/storageAccounts/blobServices/containers/write
        • Microsoft.Storage/storageAccounts/blobServices/containers/delete

      • Blob permissions

        • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read
        • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write
        • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete
        • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/move/action
        • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action

      • User delegation key permission (required for immutable containers)

        • Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action
      caution

      The generateUserDelegationKey/action permission is mandatory for immutable Azure Blob Storage. Without it, Stork cannot generate user delegation keys and backup operations to immutable containers will fail. Assign this permission before starting the installation.

      Additionally, the Managed Identity must have the minimum required Portworx Enterprise permissions for Workload Identity, typically assigned via the portworx-wli-role. These permissions grant Portworx access to Azure compute resources (disks, VMs, and node pools) and are separate from the backup location permissions listed above:

      • Microsoft.ContainerService/managedClusters/agentPools/read
      • Microsoft.Compute/disks/read
      • Microsoft.Compute/disks/write
      • Microsoft.Compute/disks/delete
      • Microsoft.Compute/virtualMachines/read
      • Microsoft.Compute/virtualMachines/write
      • Microsoft.Compute/virtualMachineScaleSets/virtualMachines/read
      • Microsoft.Compute/virtualMachineScaleSets/virtualMachines/write
      note

      The list above covers the core compute permissions required for Portworx Enterprise with Workload Identity. Depending on your environment, additional permissions may be required. For the complete list of permissions required for the custom role, see Prepare your AKS Cluster in the Portworx Enterprise documentation.

    note

    The Owner role on the Azure Storage account does not grant access to blob data. You must explicitly assign the appropriate role for your storage type (Storage Blob Data Contributor for standard storage, or the custom role for immutable storage). Without this, Stork cannot read or write backup data even if the identity has Owner-level access.

• Common requirements for both Classic and Federated mode:

  • Requirements for Portworx Backup Installation
  • Set up the installation environment

Installation workflow

Installing Portworx Backup in Federated mode involves the following high-level steps:

1. Verify Stork configuration on each application cluster

2. Generate installation specs using Portworx Central

3. Install Portworx Backup on the backup cluster

4. Monitor the installation

5. Verify pod status

Verify Stork configuration on each application cluster

Before installing Portworx Backup, verify that Stork on each application cluster is correctly configured to use Azure Managed Identity. In Federated mode, Stork authenticates directly to Azure Blob Storage using Workload Identity. No credentials are stored on the Portworx Backup server.

Azure Workload Identity for Stork is configured through the Portworx Operator as part of the Portworx Enterprise installation on each application cluster. For setup instructions, see Prepare your AKS Cluster in the Portworx Enterprise documentation. Once configured, complete the following verification steps to confirm the setup is correct before installing Portworx Backup.

note

In Federated mode, Azure Workload Identity (WLI) for Stork is configured through the StorageCluster (STC) custom resource, not through the Stork deployment manifest. The Managed Identity client ID and required labels are set in the STC spec and applied automatically by the Portworx Operator. You do not need to manually update the Stork service account or namespace.

Perform the following verification steps on each application cluster:

1. Confirm that an Azure Managed Identity is provisioned for the application cluster with the required role on the target Azure Blob Storage account:

   • Standard (non-immutable) Azure Blob Storage: verify the Storage Blob Data Contributor role is assigned.
   • Immutable Azure Blob Storage: verify the custom role (for example, portworx-backup-locked-role) is assigned with all permissions listed in the Prerequisites section, including the Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action permission. This permission is required to generate user delegation keys for immutable containers and must be configured before installation.
   note

   The Owner role on the Azure Storage account does not grant access to blob data. You must explicitly assign the appropriate role for your storage type. Without this, Stork cannot read or write backup data even if the identity has Owner-level access.

2. Verify that the Stork service account is annotated with the Managed Identity client ID:

   kubectl get serviceaccount stork -n <portworx-installed-namespace> -o jsonpath='{.metadata.annotations}'

   The output should include azure.workload.identity/client-id: <managed-identity-client-id>. If the annotation is missing, verify that the Managed Identity client ID is correctly set in the StorageCluster (STC) CR, or contact your cluster administrator.

3. Verify that the <portworx-installed-namespace> namespace is labeled to enable Azure Workload Identity:

   kubectl get namespace <portworx-installed-namespace> --show-labels

   The output should include azure.workload.identity/use=true. If the label is missing, verify the StorageCluster (STC) CR configuration or contact your cluster administrator.

   note

   Azure Workload Identity is typically enabled at the pod level via the azure.workload.identity/use: "true" pod label, which is read by the mutating webhook. In Portworx deployments, the Portworx Operator also applies this label to the <portworx-installed-namespace> namespace as part of the StorageCluster (STC) configuration, which causes the webhook to inject the required environment variables and projected volume into all pods in the namespace that includes Stork. You verify this namespace-level label in this step.

4. Verify that Stork is running:

   kubectl get pods -n <portworx-installed-namespace> -l name=stork

   All Stork pods should show a Running status.

Generate installation specs using Portworx Central

Use the Portworx Central Spec Generator to configure your Federated mode deployment. The Spec Generator guides you through the required settings and produces the Helm commands needed for installation in the next step.

note

Portworx Backup installation requires configuring passwords for various database components. Ensure that you follow the password guidelines listed in the Password Policy section.

To generate the installation specs, follow these steps:

1. Access the Portworx Central portal.

2. Enter your user credentials and review the End-User License Agreement (EULA).

   If you are new to Portworx Central, click Create account to generate your login credentials. You can also sign in using your organization's OIDC identity provider, Google, or GitHub account.

3. Click Sign In to access the portal.

4. In the Spec Details tab, provide the following values:

   1. Release Information

      • Backup Version: select the required version of Portworx Backup from the drop-down list.
      • Namespace: provide the name of the namespace where Portworx Backup will be installed (for example, central).
      • Select your environment: choose On-Premises or Cloud based on your storage environment.
      • Select Deployment Architecture: choose Federated, where backup operations are offloaded to Stork running on each shoot cluster and cloud credentials are not stored centrally in Portworx Backup. For information on installing in Classic mode, see Install Portworx Backup in Classic Mode.

   2. Configuration

      • StorageClass Name: name of the StorageClass to use for Portworx Backup persistent storage on the backup cluster. Refer to the tooltip for more details.

        note

        Defining the namespace and StorageClass is a one-time configuration step that must be completed during the initial installation. PVCs are created using the Namespace and StorageClass values defined in the SpecGen, and these values cannot be modified during an upgrade.

      • Use your OIDC: select this checkbox only if your external authorization provider is Auth0 and enter the following fields:

        • Endpoint: URL of your Auth0 domain (for example, https://your-domain.auth0.com).
        • Client ID: unique identifier for your Auth0 application.
        • Client Secret: secret key associated with your Auth0 application.

        These values can be fetched from the Auth0 web console. For more information, see Integrate External Auth Providers.

      • Use existing Prometheus: select this checkbox if you want to use your existing Prometheus stack to monitor Portworx Backup and enter the following fields:

        • Prometheus Endpoint: URL of your Prometheus server (for example, http://prometheus.example.com:9090).
        • Alertmanager Endpoint: URL of your Alertmanager server (for example, http://alertmanager.example.com:9093).
        • Prometheus secret name: name of the Kubernetes secret containing Prometheus authentication credentials.
        • Alertmanager secret name: name of the Kubernetes secret containing Alertmanager authentication credentials.
        • Use Custom email template from PX-Backup: select to upload Portworx Backup's custom email template to your pre-configured Alertmanager.

      • Proxy Configuration: select this checkbox if you have a proxy server configured in your environment and enter the following fields:

        • Unauthenticated: choose this if you have an unauthenticated proxy server:

          • HTTP Proxy: URL of the HTTP proxy server (for example, http://proxy.example.com:8080).
          • HTTPS Proxy: URL of the HTTPS proxy server (for example, https://proxy.example.com:8080).
          • NO PROXY: comma-separated list of hostnames, IPs, or domains that should bypass the proxy.

        • Authenticated/CA: choose this option if your cluster is behind a proxy server with authentication or CA:

          • Proxy configuration secret: name of the Kubernetes proxy configuration secret created in the Portworx Backup namespace.

      • Enable Telemetry: select this checkbox to enable Portworx Backup to send telemetry data to Pure1. When enabled, the Spec Generator appends --set telemetry.enabled=true to the generated Helm command. For more information, see Configure Telemetry.

   3. Under Custom Registry:

      • Non-air-gapped environment: clear the Use custom registry checkbox. Container images will be pulled directly from the Portworx image registry.
      • Air-gapped environment: select the Use custom registry checkbox and enter your private registry URL and credentials. Ensure you have completed the image pull and push steps in Set up the installation environment before this step.

   4. Database Credentials: Configure database passwords for the various database components used by Portworx Backup. For more information on password guidelines, see Password Policy.

      • MySQL Root User Password: root password for the MySQL database.
      • Postgres User Password: password for the PostgreSQL user.
      • MongoDB PX-Backup User Password: password for the MongoDB Portworx Backup user.
      • MongoDB Root User Password: root password for the MongoDB database.
      • MongoDB Replica Set Key: key used for MongoDB replica set authentication.
      • Encrypt MongoDB: select this checkbox to enable encryption at rest for the MongoDB database.
      • MongoDB Master Encryption Key: master encryption key for MongoDB encryption at rest.
      caution

      Safeguarding your encryption key is vital to prevent permanent data loss. If the key is lost, you will be unable to access your encrypted data — it is the only means of decryption and cannot be recovered. Encryption keys cannot be rotated after they are set.

5. Click Next to navigate to the Finish tab. The Finish tab displays the generated Kubernetes manifests and Helm commands that you can use in the Install Portworx Backup on the backup cluster section to complete the installation.

Install Portworx Backup on the backup cluster

You can install Portworx Backup on the backup cluster using either of the following methods. Both methods use the Helm values generated by the Portworx Central Spec Generator as described in the Generate installation specs using Portworx Central section.

• Method 1: Install using Helm — Run the generated Helm commands directly from the terminal.
• Method 2: Install using Argo CD — Use the Argo CD web console for a GitOps-based continuous delivery setup.

Method 1: Install using Helm

Run all commands in this step on the backup cluster. The commands are generated by the Portworx Central Spec Generator and are displayed in the Finish tab.

1. Create the credentials secret.
   Copy the command displayed under Step 1 and run it in your terminal to create a Kubernetes secret containing database credentials for Portworx Backup components.
   The command creates the specified namespace and a pxc-credentials secret. All sensitive data is automatically base64-encoded.

   note

   In-place upgrade from Classic mode (for example, 2.11.x) to Federated mode (3.0.x) is not supported. You can upgrade Classic mode from 2.11.x to 3.0.x, but you cannot convert an existing Classic deployment to Federated mode. Federated mode must be installed as a new deployment.

2. Add the Helm repository.
   Copy and run the command displayed under Step 2 in your terminal.

3. Install Portworx Backup.
   You can install Portworx Backup using the set command or using the values-px-central.yaml file.

   Option 1: Install using the set command

   From the Finish tab, copy and run the generated install command. As you selected Federated in the Spec Details tab, the generated command includes pxbackup.federated=true. For example:

   helm install px-central portworx/px-central \
     --namespace central \
     --create-namespace \
     --version <pxb-version> \
     --set persistentStorage.enabled=true \
     --set persistentStorage.storageClassName="<storage-class-name>" \
     --set pxbackup.enabled=true \
     --set pxbackup.federated=true \
     --set telemetry.enabled=true   # optional: remove this line if telemetry is not required

   note

   If you are deploying Portworx Backup in a cluster with Istio or Linkerd as a service mesh, append istio.enabled=true (sidecar mode only) or linkerd.enabled=true at the end of the command. The hostName parameter is mandatory if multiple applications use the same prefix (/) in Istio sidecar mode. For more information, see Configure a dedicated hostname for Portworx Backup UI with Istio.

   Option 2: Install using the values-px-central.yaml file

   1. Click the values-px-central.yaml option provided under Install using the values-px-central.yaml file. This downloads the file with all your configuration overrides, including the following Federated mode setting:

      pxbackup:
        federated: true
      telemetry:
        enabled: true   # optional: remove this line if telemetry is not required

   2. Rename the file to values-px-central-<pxb-release-version>.yaml.

   3. If you are using Istio or Linkerd, set the appropriate values in the file:

      istio:
        enabled: true
        hostName: ""
      linkerd:
        enabled: true

   4. Save the file and run:

      helm install px-central portworx/px-central \
        --namespace central \
        --create-namespace \
        --version <pxb-version> \
        -f values-px-central-<pxb-release-version>.yaml

4. Click Finish in the Portworx Central portal after you complete the installation. This activates the trial version of Portworx Backup. To activate the enterprise features, apply a Portworx Backup license.

Method 2: Install using Argo CD

You can use Argo CD to deploy Portworx Backup in a GitOps-based continuous delivery setup. The Argo CD deployment uses the same Helm values generated by the Portworx Central Spec Generator as described in the Generate installation specs using Portworx Central section.

note

When configuring Helm parameters in the Argo CD web console, ensure that pxbackup.federated=true is included in the values. The Portworx Central Spec Generator includes this parameter automatically when you select Federated as the deployment architecture in the Spec Details tab. If you paste the generated values from the Finish tab directly into Argo CD, this parameter is already set.

If you are deploying using Argo CD, also select the Argo CD deployment option in the Portworx Central Spec Generator. This sets isArgoCD=true and generates manifests compatible with Argo CD-based deployments. Without this setting, the generated values may not be suitable for use with Argo CD.

For more information on deploying Portworx Backup using Argo CD, see Deploy Portworx Backup Using Argo CD.

Monitor the installation

After running the Helm install command, monitor the post-install hook to ensure successful deployment. If the installation fails, see Retrieving health check results after a failed install or upgrade to diagnose the issue.

# Check post-install hook status
kubectl get pod --namespace <pxb-namespace> -l job-name=pxcentral-post-install-hook -o wide | awk '{print $1, $3}' | grep -iv error

# Monitor until completion
kubectl get job pxcentral-post-install-hook -n <pxb-namespace> -w

Expected output:

pxcentral-post-install-hook-xxxxx   Completed

Verify pod status

Ensure all Portworx Backup components are running on the backup cluster:

kubectl get pods -n <pxb-namespace>

All pods should be in Running or Completed state.

Next steps

• Set up Portworx Backup for Operations
• Onboard clusters and configure backup locations — After installation, add your Gardener shoot clusters to Portworx Backup and assign backup locations to each cluster. For more information, see:
  • Manage Clusters (Federated mode)
  • Configure Backup Locations (Federated Mode)