Rejoin a decommissioned Portworx node back to the GKE cluster
This document provides instructions for rejoining a previously decommissioned node with its original cluster.
Ensure the node is decommissioned from Portworx
If the node was previously part of a Portworx cluster, you must first ensure you followed steps to decommission it from the cluster. The Decommission a Node page has detailed steps for this.
Ensure the Portworx services are no longer running
You must ensure that the Portworx services are completely disabled from your system. You can check the status of the portworx systemd
service using the following systemctl
command:
sudo systemctl status portworx
If the systemd
service is still running, enter the following commands to stop and disable it:
sudo systemctl stop portworx
sudo systemctl disable portworx
sudo rm -f /etc/systemd/system/portworx*
grep -q '/opt/pwx/oci /opt/pwx/oci' /proc/self/mountinfo && sudo umount /opt/pwx/oci
Clean up Portworx metadata on the node
Portworx also stores local metadata on the node to fingerprint its identity. The method you used to decommission the node determines the steps you must follow to clean up local Portworx metadata.
Remove metadata using pxctl
If pxctl is still on your node, enter the following pxctl service
command to remove local metadata using pxctl:
pxctl service node-wipe --all
Remove metadata using wipefs
If pxctl has already been removed from your node, you must manually find and wipe Portworx disks:
-
Enter the following
blkid
andgrep
command to list the disks that Portworx was using:blkid | grep pxpool
-
For each of the disks output in step one, enter the
wipefs
command with the-af
options and the<disk-name>
to wipe the Portworx metadata:wipefs -af <disk-name>
Example
wipefs -af /dev/sdf
Restart the Portworx pod on your node by cleaning up labels
Once you have completely decommissioned your node and wiped it of Portworx data, you're ready to re-add it to your cluster. Enter the following kubectl label nodes
commands, replacing <node>
with the Kubernetes node name of this node:
kubectl label node <node-name> px/service=restart --overwrite
kubectl label node <node-name> px/enabled=true --overwrite
Once you've entered these commands, Portworx will start on this node and rejoin the cluster as a new node.