Collecting diagnostics


About diagnostics

A Portworx diagnostics bundle (often referred to as just “diags”) is a support bundle that contains all the information our support team would need to diagnose an issue in your cluster. It mainly contains

  • PX journald logs
  • Output from common PX CLI command that provide details about cluster, nodes and volumes
  • Basic information about the OS of the cluster nods
  • Stack and heap files of PX processes
  • Alerts generated by the cluster
  • Cores or traces from PX processes (if found)

Auto-uploading diags using Pure1 integration

With PX 2.8.0 or above, Portworx can now automatically upload it’s diags to Pure Storage’s call home service called Pure1. The primary benefits of this are

  • End users don’t have to copy out diag bundles from their cluster nodes and upload them to support tickets
  • When nodes run into issues, they automatically collect diags and call home them. This allows Portworx support to get ahead of the issues and reduces the time to resolution.

When working on a support case, provide your Cluster UUID to Portworx support. They will be able to retrieve your diags from Pure1.

What is my cluster’s UUID? If you are using operator-based install, you can find this here (See “CLUSTER UUID” column):

kubectl get storagecluster -n kube-system
NAME                                              CLUSTER UUID                           STATUS   VERSION           AGE
px-cluster-2ec933a4-29be-440c-b5b1-ec53d3ba5b39   199ccacd-4253-4e57-9a3d-b2249dff3501   Online   01d934d_d03a058   26h

Another place to find your cluster’s UUID is in the output of /opt/pwx/bin/pxctl status.

pxctl status | grep 'Cluster UUID'
        Cluster UUID: 199ccacd-4253-4e57-9a3d-b2249dff3501

Pre-requisites

  • Portworx 2.8.0 or above
  • For Operator based installations, Portworx Operator 1.5.0 or above
  • For DaemonSet based installs, ensure that you have generated the Portworx spec using https://central.portworx.com/
  • Outbound access to the internet to allow connection to Pure1

Enabling Pure1 integration

Enabling telemetry adds a new telemetry sidecar container to Portworx pods. This container is responsible for uploading PX diagnostics to Pure1.

Fresh installs

This is automatically enabled for all new clusters with PX 2.8.0 or above. See Pre-requisites.

Upgrades

If you are upgrading from PX version prior to 2.8.0, follow the Enabling Pure1 integration for upgrades page for instructions.

Disabling Pure1 integration

Operator based install

To disable the telemetry integration with Pure1, you have to add the following section in your StorageCluster spec

spec:
  monitoring:
    telemetry:
      enabled: false 
Note that above is in addition to any existing fields in your StorageCluster spec

This will remove the telemetry container from Portworx pod.

DaemonSet based install

To disable telemetry integration for DaemonSet based install, simply remove the telemetry container from the Portworx DaemonSet.

Collecting diagnostics

Portworx diagnostics are collected in primarily 2 ways:

1. On demand

This is done via the pxctl CLI. The most common command used here is:

/opt/pwx/bin/pxctl service diags -a

This generates the diagnostics bundle on the node. If the Pure1 telemetry is enabled, it will be automatically uploaded to Pure1.

See Generate a complete diagnostics package for the entire CLI syntax.

2. On crash

If a Portworx process runs into an issue on a node, it will automatically collect diagnostics. If the Pure1 telemetry is enabled, it will be automatically uploaded to Pure1.



Last edited: Wednesday, Oct 27, 2021