Skip to main content
Version: 3.5

Install Portworx on Mirantis Kubernetes Engine

You can install Portworx on a Mirantis Kubernetes Engine (MKE) cluster to enable enterprise-grade cloud-native storage for your Kubernetes workloads. Portworx supports standard Kubernetes deployments running on MKE. Portworx integrates natively with MKE, ensuring persistent storage for demanding applications.

Portworx installation on an MKE cluster is managed via Kubernetes manifests, generated through Portworx Central. You can apply a Portworx Operator and StorageCluster manifest to your MKE cluster, which automates and orchestrates the installation across all nodes. The operator-based deployment is recommended for ease of management and updates.

Prerequisites​

In addition to the System Requirements, ensure that your cluster meets the following requirements before installing Portworx Enterprise:

  • Provision virtual machines and use them as nodes to create a Kubernetes cluster managed by MKE.
    For more information, see Mirantis documentation.
  • Allocate a dedicated disk for the internal KVDB.
  • Configure the KVDB device on only three nodes, and assign it a unique device name on each of those nodes.

The following collection of tasks describe how to install Portworx on an MKE cluster:

Complete all the tasks to install Portworx.

Prepare your environment​

Portworx Service Accounts, which are non-admin, should be granted access to use privileged attributes on Kubernetes Pods. This enables Portworx to create/execute tasks that would ordinarily require administrators or cluster-admins permissions to execute.

Portworx needs access to the following privileged attributes:

priv_attributes_allowed_for_service_accounts = ["hostBindMounts", "privileged", "kernelCapabilities", "hostNetwork"]

You must also configure the following Service Accounts:

priv_attributes_service_accounts = ["<pxNamespace>:portworx-operator","<pxNamespace>:portworx","<pxNamespace>:autopilot", "<pxNamespace>:px-csi", "<pxNamespace>:portworx-pvc-controller"]

Replace <pxNamespace> with namespace where Portworx is installed.

You can grant Portworx access to use privileged attributes by adding privileged attributes using one of the following options:

  • When you install a new MKE cluster: Grant Portworx access to use privileged attributes during the MKE cluster installation process. For more information, see Mirantis documentation.
  • Modify an existing MKE cluster: Update the configuration of your current MKE cluster to allow Portworx Service Accounts to use the required privileged attributes. For more information, see Mirantis documentation.

Generate Portworx Specification​

  1. Sign in to the Portworx Central console.
    The system displays the Welcome to Portworx Central! page.

  2. In the Portworx Enterprise section, select Generate Cluster Spec.
    The system displays the Generate Spec page.

  3. From the Portworx Version dropdown menu, select the Portworx version to install.

  4. From the Platform dropdown menu, select one of the following depending on your environment:

    • DAS/SAN
    • Pure FlashArray
    • vSphere
    note
    • For DAS/SAN, you must have pre-provisioned disks.
    • For Pure FlashArray or vSphere, you can specify the disk when you generate the Portworx spec.
      After you apply the Portworx StorageCluster spec, these disks will be created in your environment.

  5. From the Distribution Name dropdown menu, select None.

  6. (Optional) To customize the configuration options and generate a custom specification, click Customize and perform the following steps:

    note

    To continue without customizing the default configuration or generating a custom specification, proceed to Step 7.

  • Basic tab:

    1. To use an existing etcd cluster, do the following:
      1. Select the Your etcd details option.
      2. In the field provided, enter the host name or IP and port number. For example, http://test.com.net:1234.
        To add another etcd cluster, click the + icon.
        note

        You can add up to three etcd clusters.

      3. Select one of the following authentication methods:
        • Disable HTTPS – To use HTTP for etcd communication.
        • Certificate Auth – To use HTTPS with an SSL certificate.
          For more information, see Secure your etcd communication.
        • Password Auth – To use HTTPS with username and password authentication.
    2. To use an internal Portworx-managed key-value store (kvdb), do the following:
      1. Select the Built-in option.
      2. To enable TLS encrypted communication among KVDB nodes and between Portworx nodes and the KVDB cluster, select the Enable TLS for internal kvdb checkbox.
      3. If your cluster does not already have a cert-manager, select the Deploy Cert-Manager for TLS certificates checkbox.
    3. Select Next.

  • Storage tab:

    note

    The following steps apply only if you selected DAS/SAN as the storage platform. If you are using Pure FlashArray or vSphere, follow the corresponding procedure for your platform. For more information, see Install Portworx with Pure Storage FlashArray or Installation on Non-Air-Gapped vSphere Kubernetes Cluster.

    1. To enable Portworx to use all available, unused, and unmounted drives on the node, do the following:

      1. Select the Automatically scan disks option.
      2. From the Default IO Profile dropdown menu, select Auto.
        This enables Portworx to automatically choose the best I/O profile based on detected workload patterns.
      3. Select the Use unmounted disks even if they have a partition or filesystem on it. Portworx will never use a drive or partition that is mounted checkbox to use unmounted disks, even if they contain a partition or filesystem.
        Portworx will not use any mounted drive or partition.

    2. To manually specify the drives on the node for Portworx to use, do the following:

      1. Select the Manually specify disks option.
      2. In the Drive/Device field, specify the block drive(s) that Portworx uses for data storage.
        To add another block drive, click the + icon.
      3. (Optional) In the Pool Label field, assign a custom label in key:value format to identify and categorize storage pools.

    3. Select PX-Store Version: (Optional) To designate PX-StoreV1 as the datastore, select PX-StoreV1 if you are on platform vSphere or Pure FlashArray.

      In case of DAS/SAN platform, to designate PX-StoreV1 as the datastore, clear the PX-StoreV2 checkbox. By default, the system selects PX-StoreV2 as the datastore.

    4. For PX-StoreV2, in Metadata Path field, enter a pre-provisioned path for storing the Portworx metadata.
      The path must be at least 64 GB in size.

    5. From the Journal Device dropdown menu, select one of the following:

      • None – To use the default journaling setting.
      • Auto – To automatically allocate journal devices.
      • Custom – To manually enter a journal device path.
        Enter the path of the journal device in the Journal Device Path field.

    6. Skip KVDB device - This checkbox is selected by default and appears only if you choose the Built-in option in the Basic tab.
      Keep it selected to use the same device for KVDB and storage I/O. This configuration is suitable for test or development clusters but not recommended for production clusters. For production clusters, clear the checkbox and provide a separate device to store internal KVDB data. This separates KVDB I/O from storage I/O and improves performance.

    7. KVDB device - Enter the block device path to be used exclusively for KVDB data.
      This field appears only if you clear the Skip KVDB device checkbox. The KVDB device must be present on at least three nodes in the cluster to ensure high availability.

      note

      To restrict Portworx to run internal KVDB only on specific nodes, label those nodes with:

      kubectl label nodes node1 node2 node3 px/metadata-node=true

    8. Click Next.

  • Network tab:

    1. In the Interface(s) section, do the following:
      1. Enter the Data Network Interface to be used for data traffic.
      2. Enter the Management Network Interface to be used for management traffic.
    2. In the Advanced Settings section, do the following:
      1. Enter the Starting port for Portworx services.
    3. Select Next.

  • Deployment tab:

    1. In the Kubernetes Distribution section, under Are you running on either of these?, choose the Kubernetes distribution.
    2. In the Component Settings section:
      1. Select the Enable Stork checkbox to enable Stork.
      2. Select the Enable Monitoring checkbox to enable Prometheus-based monitoring of Portworx components and resources.
      3. To configure how Prometheus is deployed and managed in your cluster, choose one of the following:
        • Portworx Managed - To enable Portworx to install and manage Prometheus and Operator automatically.
          Ensure that no another Prometheus Operator instance already running on the cluster.
        • User Managed - To manage your own Prometheus stack.
          You must enter a valid URL of the Prometheus instance in the Prometheus URL field.
      4. Select the Enable Autopilot checkbox to enable Portworx Autopilot.
        For more information on Autopilot, see Expanding your Storage Pool with Autopilot.
      5. Select the Enable Telemetry checkbox to enable telemetry in the StorageCluster spec.
        For more information, see Enable Pure1 integration for upgrades on an MKE cluster.
      6. Enter the prefix for the Portworx cluster name in the Cluster Name Prefix field.
      7. Select the Secrets Store Type from the dropdown menu to store and manage secure information for features such as CloudSnaps and Encryption.
    3. In the Environment Variables section, enter name-value pairs in the respective fields.
    4. In the Registry and Image Settings section:
      1. Enter the Custom Container Registry Location to download the Docker images.
      2. Enter the Kubernetes Docker Registry Secret that serves as the authentication to access the custom container registry.
      3. From the Image Pull Policy dropdown menu, select Default, Always, IfNotPresent, or Never.
        This policy influences how images are managed on the node and when updates are applied.
    5. In the Security Settings section, select the Enable Authorization checkbox to enable Role-Based Access Control (RBAC) and secure access to storage resources in your cluster.
    6. Click Finish.
    7. In the summary page, enter a name for the specification in the Spec Name field, and tags in the Spec Tags field.
    8. Click Download .yaml to download the yaml file with the customized specification or Save Spec to save the specification.
  1. Click Save & Download to generate the specification.

Deploy Portworx Operator​

Use the Operator specifications you generated in the Generate Portworx Specification section, and deploy Portworx Operator by running the following command.

kubectl apply -f 'https://install.portworx.com/<version-number>?comp=pxoperator'
serviceaccount/portworx-operator created
podsecuritypolicy.policy/px-operator created
clusterrole.rbac.authorization.k8s.io/portworx-operator created
clusterrolebinding.rbac.authorization.k8s.io/portworx-operator created
deployment.apps/portworx-operator created

Verify that the Operator is running.

kubectl get pods -lname=portworx-operator -A
kube-system         portworx-operator-584ccd74c7-7bhkt                   1/1     Running   0              3m48s

Deploy StorageCluster​

Use the StorageCluster specifications you generated in the Generate Portworx Specification section, and deploy StorageCluster by running the following command.

kubectl apply -f 'https://install.portworx.com/<version-number>?operator=true&mc=false&kbver=&b=true&c=px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-8dfd338e915b&stork=true&csi=true&mon=true&tel=false&st=k8s&promop=true'
storagecluster.core.libopenstorage.org/px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-8dfd338e915b created

Monitor Portworx nodes​

  1. Enter the following kubectl get command and wait until all Portworx nodes show as Ready or Online in the output:

    kubectl -n <px-namespace> get storagenodes -l name=portworx
    NAME                 ID                                     STATUS   VERSION          AGE
    username-k8s1-node0   xxxxxxxx-xxxx-xxxx-xxxx-43cf085e764e   Online   2.11.1-3a5f406   4m52s
    username-k8s1-node1   xxxxxxxx-xxxx-xxxx-xxxx-4597de6fdd32   Online   2.11.1-3a5f406   4m52s
    username-k8s1-node2   xxxxxxxx-xxxx-xxxx-xxxx-e2169ffa111c   Online   2.11.1-3a5f406   4m52s

  2. Enter the following kubectl describe command with the NAME of one of the Portworx nodes you retrieved in Step 1 to show the current installation status for individual nodes:

    kubectl -n <px-namespace> describe storagenode <portworx-node-name>
    ...
    Events:
        Type     Reason                             Age                     From                  Message
        ----     ------                             ----                    ----                  -------
        Normal   PortworxMonitorImagePullInPrgress  7m48s                   portworx, k8s-node-2  Portworx image portworx/px-enterprise:2.10.1.1 pull and extraction in progress
        Warning  NodeStateChange                    5m26s                   portworx, k8s-node-2  Node is not in quorum. Waiting to connect to peer nodes on port 9002.
        Normal   NodeStartSuccess                   5m7s                    portworx, k8s-node-2  PX is ready on this node
    note
    • In your output, the image pulled will differ based on your chosen Portworx license type and version.
    • For Portworx Enterprise, the default license activated on the cluster is a 30 day trial that you can convert to a SaaS-based model or a generic fixed license.

Verify Portworx Pod Status​

Enter the following command to list and filter the results for Portworx pods and specify the namespace where you have deployed Portworx:

kubectl get pods -n <px-namespace> -o wide | grep -e portworx -e px
NAME                                                    READY   STATUS    RESTARTS         AGE     IP                NODE                   NOMINATED NODE   READINESS GATES
portworx-api-774c2                                      1/1     Running   0                2m55s   192.168.121.196   username-k8s1-node0    <none>           <none>
portworx-api-t4lf9                                      1/1     Running   0                2m55s   192.168.121.99    username-k8s1-node1    <none>           <none>
portworx-api-dvw64                                      1/1     Running   0                2m55s   192.168.121.99    username-k8s1-node2    <none>           <none>
portworx-kvdb-94bpk                                     1/1     Running   0                4s      192.168.121.196   username-k8s1-node0    <none>           <none>
portworx-kvdb-8b67l                                     1/1     Running   0                10s     192.168.121.196   username-k8s1-node1    <none>           <none>
portworx-kvdb-fj72p                                     1/1     Running   0                30s     192.168.121.196   username-k8s1-node2    <none>           <none>
portworx-operator-58967ddd6d-kmz6c                      1/1     Running   0                4m1s    10.244.1.99       username-k8s1-node0    <none>           <none>
prometheus-px-prometheus-0                              2/2     Running   0                2m41s   10.244.1.105      username-k8s1-node0    <none>           <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-9gs79   2/2     Running   0                2m55s   192.168.121.196   username-k8s1-node0    <none>           <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-vpptx   2/2     Running   0                2m55s   192.168.121.99    username-k8s1-node1    <none>           <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-bxmpn   2/2     Running   0                2m55s   192.168.121.191   username-k8s1-node2    <none>           <none>
px-csi-ext-868fcb9fc6-54bmc                             4/4     Running   0                3m5s    10.244.1.103      username-k8s1-node0    <none>           <none>
px-csi-ext-868fcb9fc6-8tk79                             4/4     Running   0                3m5s    10.244.1.102      username-k8s1-node2    <none>           <none>
px-csi-ext-868fcb9fc6-vbqzk                             4/4     Running   0                3m5s    10.244.3.107      username-k8s1-node1    <none>           <none>
px-prometheus-operator-59b98b5897-9nwfv                 1/1     Running   0                3m3s    10.244.1.104      username-k8s1-node0    <none>           <none>

Note the name of a px-cluster pod. You will run pxctl commands from these pods in Verify Portworx Cluster Status.

Verify Portworx Cluster Status​

You can find the status of the Portworx cluster by running pxctl status commands from a pod.
Enter the following kubectl exec command, specifying the pod name you retrieved in Verify Portworx Pod Status:

kubectl exec <pod-name> -n <px-namespace> -- /opt/pwx/bin/pxctl status
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Status: PX is operational
Telemetry: Disabled or Unhealthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e
    IP: 192.168.121.99 
    Local Storage Pool: 1 pool
    POOL    IO_PRIORITY     RAID_LEVEL      USABLE  USED    STATUS  ZONE    REGION
    0       HIGH            raid0           3.0 TiB 10 GiB  Online  default default
    Local Storage Devices: 3 devices
    Device  Path            Media Type              Size            Last-Scan
    0:1     /dev/vdb        STORAGE_MEDIUM_MAGNETIC 1.0 TiB         14 Jul 22 22:03 UTC
    0:2     /dev/vdc        STORAGE_MEDIUM_MAGNETIC 1.0 TiB         14 Jul 22 22:03 UTC
    0:3     /dev/vdd        STORAGE_MEDIUM_MAGNETIC 1.0 TiB         14 Jul 22 22:03 UTC
    * Internal kvdb on this node is sharing this storage device /dev/vdc  to store its data.
    total           -       3.0 TiB
    Cache Devices:
        * No cache devices
Cluster Summary
    Cluster ID: px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d
    Cluster UUID: xxxxxxxx-xxxx-xxxx-xxxx-6f3fd5522eae
    Scheduler: kubernetes
    Nodes: 3 node(s) with storage (3 online)
    IP              ID                                      SchedulerNodeName       Auth            StorageNode     Used    Capacity        Status  StorageStatus       Version         Kernel                  OS
    192.168.121.196 xxxxxxxx-xxxx-xxxx-xxxx-fad8c65b8edc    username-k8s1-node0      Disabled        Yes             10 GiB  3.0 TiB         Online  Up 2.11.0-81faacc   3.10.0-1127.el7.x86_64  CentOS Linux 7 (Core)
    192.168.121.99  xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e    username-k8s1-node1      Disabled        Yes             10 GiB  3.0 TiB         Online  Up (This node)      2.11.0-81faacc  3.10.0-1127.el7.x86_64  CentOS Linux 7 (Core)
    192.168.121.191 xxxxxxxx-xxxx-xxxx-xxxx-19d45b4c541a    username-k8s1-node2      Disabled        Yes             10 GiB  3.0 TiB         Online  Up  2.11.0-81faacc  3.10.0-1127.el7.x86_64  CentOS Linux 7 (Core)
Global Storage Pool        
    Total Used      :  30 GiB
    Total Capacity  :  9.0 TiB

Status displays PX is operational when the cluster is running as expected. If the cluster is using the PX-StoreV2 datastore, the StorageNode entries for each node displays Yes(PX-StoreV2).

Verify pxctl Cluster Provision Status​

  1. Access the Portworx CLI.

  2. Run the following command to find the storage cluster:

    kubectl -n <px-namespace> get storagecluster
    NAME                                              CLUSTER UUID                           STATUS   VERSION   AGE
    px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d   xxxxxxxx-xxxx-xxxx-xxxx-6f3fd5522eae   Online   2.11.0    10m

    The status must display the cluster is Online.

  3. Run the following command to find the storage nodes:

    kubectl -n <px-namespace> get storagenodes
    NAME                  ID                                     STATUS   VERSION          AGE
    username-k8s1-node0   xxxxxxxx-xxxx-xxxx-xxxx-fad8c65b8edc   Online   2.11.0-81faacc   11m
    username-k8s1-node1   xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e   Online   2.11.0-81faacc   11m
    username-k8s1-node2   xxxxxxxx-xxxx-xxxx-xxxx-19d45b4c541a   Online   2.11.0-81faacc   11m

    The status must display the nodes are Online.

  4. Verify the Portworx cluster provision status by running the following command.
    Specify the pod name you retrieved in Verify Portworx Pod Status.

    kubectl exec <px-pod> -n <px-namespace> -- /opt/pwx/bin/pxctl cluster provision-status
    NODE					                NODE STATUS	 POOL						              POOL STATUS  IO_PRIORITY	SIZE	AVAILABLE	USED   PROVISIONED ZONE REGION	RACK
    0c99e1f2-9d49-xxxx-xxxx-xxxxxxxxxxxx	Up		    0 ( 8ec9e6aa-7726-xxxx-xxxx-xxxxxxxxxxxx )	Online		HIGH		32 GiB	32 GiB		33 MiB	0 B		default	default	default
    1e89102f-0510-xxxx-xxxx-xxxxxxxxxxxx	Up		    0 ( 06fcc73a-7e2f-xxxx-xxxx-xxxxxxxxxxxx )	Online		HIGH		32 GiB	32 GiB		33 MiB	0 B		default	default	default
    24508311-e2fe-xxxx-xxxx-xxxxxxxxxxxx	Up		    0 ( 58ab2e3f-a22e-xxxx-xxxx-xxxxxxxxxxxx )	Online		HIGH		32 GiB	32 GiB		33 MiB	0 B		default	default	default

What to do next​

Create a PVC. For more information, see Create your first PVC.