Reactions to etcd recovery
The following table summarizes how Portworx will respond to an etcd disaster and its recovery from a previous snapshot.
| Portworx state when snapshot was taken | Portworx state just before disaster | Portworx state after disaster recovery |
|---|---|---|
| Portworx running with few volumes | No Portworx or application activity | Portworx is back online. Volumes are intact. No disruption. |
| Portworx running with few volumes | New volumes created | Portworx is back online. New Volumes are lost. |
| Portworx volumes were not in use by application. (Volumes are not attached) | Volumes are now in use by application (Volumes are attached) | Portworx is back online. The volume which was supposed to be attached is in detached state. Application is in CrashLoopBackOff state. Potentially could lead to data loss. |
| Portworx volumes were in use by application | Volume are now not in use by application | Volumes which are not in use by the application still stay attached. No data loss involved. |
| All Portworx nodes are up | No Portworx Activity | All the expected nodes are still Up |
| All Portworx nodes are up | A few nodes go down which have volume replica. Current Set changes. | Potentially could lead to data loss/corruption. Current Set is not in sync with what the storage actually has and when Portworx comes back up it might lead to data corruption |
| A Portworx node with replica is down. The node is not in current set. | The node is now online and in Current Set. | Portworx volume starts with older current set, but eventually gets updated current set. No data loss involved. |
Last edited: Friday, Apr 7, 2023
Questions? Visit the Portworx forum.