TLS Certificate

Description

TLS setup for px-backup on OpenShift with pxb. This guide explains how to enable TLS for the px-backup gRPC server, generate a certificate that works for both internal and external traffic, configure the OpenShift Route, verify readiness/connectivity, and use px pxb with TLS. It follows the same structure and conventions as backup.md and is copy/paste‑friendly.

Usage

Use px pxb with TLS (one-off, per command)

px pxb version -e <host:port> [--tls-certificate </path/to/ca.crt>]
px pxb get cluster -e <host:port> [--tls-certificate </path/to/ca.crt>]

Persist endpoint and CA certificate in config

px pxb set config --px-backup-api-url <host:port> --tls-certificate-path </path/to/ca.crt>

Use environment variable for endpoint (optional)

export PXB_GRPC_ENDPOINT="<host:port>"

Examples

What you will achieve

• TLS enabled on the px-backup gRPC server
• Certificate valid for both internal (in-pod) and external (Route) traffic
• Reliable readiness and Route forwarding
• Working px pxb connectivity over TLS
• Troubleshooting steps for common issues

Prerequisites

• OpenShift cluster with px-backup running in namespace px-backup
• oc CLI access
• Ability to edit the px-backup Deployment and manage secrets
• A passthrough Route to the gRPC Service port 10002

Step 1: Ensure Deployment has TLS flags and secret mount

Edit the px-backup Deployment and verify:

• Volume mount at /etc/tls (readOnly) from secret grpc-tls-external
• Container command contains:
  • --tls-cert-file=/etc/tls/tls.crt
  • --tls-key-file=/etc/tls/tls.key

These flags enable TLS in the server.

Step 2: Generate the certificate that works everywhere

The internal REST→gRPC TLS client verifies the server name using the listener address, which is typically [::]:10002 on dual‑stack. Without IP SAN ::, readiness will fail.

Required SANs:

• DNS: your Route host (e.g., px-backup-grpc.apps.example.com)
• DNS: localhost
• IP: 127.0.0.1
• IP: ::1
• IP: :: (important)

Script (run on your workstation). Replace HOST with your Route host and NS with your namespace.

#!/usr/bin/env bash
set -euo pipefail

HOST="<HOST>"
NS="px-backup"
DAYS_CA=3650
DAYS_SERVER=825

mkdir -p ~/ssl-demo && cd ~/ssl-demo

echo "==> Generate local CA (one-time; reuse if present)"
if [[ ! -f localCA.key || ! -f localCA.crt ]]; then
  openssl genrsa -out localCA.key 4096
  openssl req -x509 -new -nodes -key localCA.key -sha256 -days "$DAYS_CA" \
    -subj "/CN=Local Dev CA" -out localCA.crt
else
  echo "    Reusing existing localCA.key/crt"
fi

echo "==> Write OpenSSL config with SANs"
cat > openssl.cnf <<EOF
[ req ]
default_bits       = 2048
prompt             = no
default_md         = sha256
req_extensions     = req_ext
distinguished_name = dn

[ dn ]
CN = ${HOST}

[ req_ext ]
subjectAltName = @alt_names
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ alt_names ]
DNS.1 = ${HOST}
DNS.2 = localhost
IP.1  = 127.0.0.1
IP.2  = ::1
IP.3  = ::
EOF

echo "==> Generate server key + CSR"
openssl genrsa -out "${HOST}.key" 2048
openssl req -new -key "${HOST}.key" -out "${HOST}.csr" -config openssl.cnf

echo "==> Sign server CSR with local CA"
openssl x509 -req -in "${HOST}.csr" -CA localCA.crt -CAkey localCA.key -CAcreateserial \
  -out "${HOST}.crt" -days "$DAYS_SERVER" -sha256 -extensions req_ext -extfile openssl.cnf

echo "==> Verify SANs"
openssl x509 -in "${HOST}.crt" -noout -text | sed -n '/Subject Alternative Name/,+1p'

echo "==> Update K8s secret and roll out"
oc -n "${NS}" delete secret grpc-tls-external --ignore-not-found
oc -n "${NS}" create secret tls grpc-tls-external \
  --cert="${HOST}.crt" \
  --key="${HOST}.key"

oc -n "${NS}" rollout restart deploy/px-backup
oc -n "${NS}" rollout status deploy/px-backup

echo "==> External TLS test"
echo "openssl s_client -connect ${HOST}:443 -servername ${HOST} -showcerts -CAfile ~/ssl-demo/localCA.crt | sed -n '1,30p'"

Step 3: Verify Route forwards to gRPC

• Route must be passthrough and target the Service for port 10002 (gRPC)
• Check endpoints:

oc -n px-backup get endpoints <grpc-service> -o wide

• Test external TLS:

openssl s_client -connect "${HOST}:443" -servername "${HOST}" -showcerts -CAfile ~/ssl-demo/localCA.crt

Step 4: Readiness probe sanity

With the above SANs, the default HTTP readiness on /v1/health (port 10001) should pass. If you prefer, you can use a TCP or exec probe to decouple from REST→gRPC.

Step 5: Use px pxb over TLS

Three ways to set the endpoint:

• Flag (one‑off):

px pxb version -e "${HOST}:443" --tls-certificate ~/ssl-demo/localCA.crt
px pxb get cluster -e "${HOST}:443" --tls-certificate ~/ssl-demo/localCA.crt

• Environment:

export PXB_GRPC_ENDPOINT="${HOST}:443"

• Persisted config:

px pxb set config --px-backup-api-url "${HOST}:443" --tls-certificate-path ~/ssl-demo/localCA.crt
px pxb version
px pxb get cluster

If authentication is enabled, login once:

px pxb login --server "https://<pxc-auth-host>"
px pxb status config

How we tested (exact commands)

• Pod logs:

oc -n px-backup logs deploy/px-backup | egrep -i "TLS enabled|gRPC Server ready|REST Gateway started"

• In-cluster health (debug pod):

POD_IP=$(oc -n px-backup get pod -l app=px-backup -o jsonpath='{.items[0].status.podIP}')
oc -n px-backup run curl --image=curlimages/curl -it --rm --restart=Never -- sh -c 'curl -v --max-time 5 http://'$POD_IP':10001/v1/health'

• External TLS via Route:

openssl s_client -connect "${HOST}:443" -servername "${HOST}" -showcerts -CAfile ~/ssl-demo/localCA.crt

• pxb:

px pxb version -e "${HOST}:443" --tls-certificate ~/ssl-demo/localCA.crt
px pxb get cluster -e "${HOST}:443" --tls-certificate ~/ssl-demo/localCA.crt

Flags for commands and sub-commands

Global Flags Used (pxb)

Flag	Short	Type	Description
--endpoint	-e	string	PX‑Backup gRPC endpoint (host:port). Can also be set via PXB_GRPC_ENDPOINT env var, or persisted in config
--tls-certificate		string	Path to CA certificate file used to validate server TLS (one‑off per command)

Config Flags (persist settings)

Flag	Short	Type	Description
--px-backup-api-url		string	Persist the PX‑Backup gRPC endpoint (host:port)
--tls-certificate-path		string	Persist the path to the CA certificate used for TLS verification

Notes

• Endpoint resolution priority (pxb):
  • CLI flag -e/--endpoint > environment variable PXB_GRPC_ENDPOINT > config value px_backup_api_url
• TLS certificate resolution priority (pxb):
  • CLI flag --tls-certificate > config value tls_certificate_path
• The certificate file is used as the CA bundle for verifying the server; if omitted, px pxb dials without TLS.
• For OpenShift Routes, use passthrough termination and target the Service that exposes gRPC on port 10002.
• Certificate SANs must include your Route host, localhost, 127.0.0.1, ::1, and :: to satisfy in‑pod readiness checks for dual‑stack listeners.

Troubleshooting

• 503 with body mentioning valid for 127.0.0.1, ::1, not ::
  • Add IP SAN :: to the server certificate and redeploy.
• no peer certificate available from openssl s_client
  • Pod not Ready or Route pointing to wrong service/port. Ensure Route → Service targetPort 10002 and pod is Ready.
• px pxb “hangs” then errors
  • Likely connecting to wrong endpoint/port or SNI mismatch. Use ${HOST}:443 (hostname), provide the CA file, and verify Route forwards to 10002.

Output Format Support

• Not applicable: this is a practical setup guide aggregating multiple commands; output options are per individual command.

Authentication and Context

• If authentication is enabled, obtain a token using px pxb login (see login-logout.md) and verify status via px pxb status config.
• You can persist both the gRPC endpoint and TLS CA path using px pxb set config (see config.md).