Installation on Mirantis Kubernetes Engine
You can install Portworx on a Mirantis Kubernetes Engine (MKE) cluster to enable enterprise-grade cloud-native storage for your Kubernetes workloads. Portworx supports standard Kubernetes deployments running on MKE. Portworx integrates natively with MKE, ensuring persistent storage for demanding applications.
Portworx installation on an MKE cluster is managed via Kubernetes manifests, generated through Portworx Central. You can apply a Portworx Operator and StorageCluster manifest to your MKE cluster, which automates and orchestrates the installation across all nodes. The operator-based deployment is recommended for ease of management and updates.
Prerequisites
In addition to the System Requirements, ensure that your cluster meets the following requirements before installing Portworx Enterprise:
- Provision virtual machines and use them as nodes to create a Kubernetes cluster managed by MKE.
For more information, see Mirantis documentation. - Allocate a dedicated disk for the internal KVDB.
- Configure the KVDB device on only three nodes, and assign it a unique device name on each of those nodes.
The following collection of tasks describe how to install Portworx on an MKE cluster:
- Prepare Your Environment
- Generate Portworx Specification
- Deploy Portworx Operator
- Deploy StorageCluster
- Monitor Portworx Nodes
- Verify Portworx Pod Status
- Verify Portworx Cluster Status
- Verify pxctl Cluster Provision Status
Complete all the tasks to install Portworx.
Prepare your environment
Portworx Service Accounts, which are non-admin, should be granted access to use privileged attributes on Kubernetes Pods. This enables Portworx to create/execute tasks that would ordinarily require administrators or cluster-admins permissions to execute.
Portworx needs access to the following privileged attributes:
priv_attributes_allowed_for_service_accounts = ["hostBindMounts", "privileged", "kernelCapabilities", "hostNetwork"]
You must also configure the following Service Accounts:
priv_attributes_service_accounts = ["<pxNamespace>:portworx-operator","<pxNamespace>:portworx","<pxNamespace>:autopilot", "<pxNamespace>:px-csi", "<pxNamespace>:portworx-pvc-controller"]
Replace <pxNamespace>
with namespace where Portworx is installed.
You can grant Portworx access to use privileged attributes by adding privileged attributes using one of the following options:
- When you install a new MKE cluster: Grant Portworx access to use privileged attributes during the MKE cluster installation process. For more information, see Mirantis documentation.
- Modify an existing MKE cluster: Update the configuration of your current MKE cluster to allow Portworx Service Accounts to use the required privileged attributes. For more information, see Mirantis documentation.
Generate Portworx Specification
-
Sign in to the Portworx Central console.
The system displays the Welcome to Portworx Central! page. -
In the Portworx Enterprise section, select Generate Cluster Spec.
The system displays the Generate Spec page. -
From the Portworx Version dropdown menu, select the Portworx version to install.
-
From the Platform dropdown menu, select one of the following depending on your environment:
- DAS/SAN
- Pure FlashArray
- vSphere
note- For DAS/SAN, you must have pre-provisioned disks.
- For Pure FlashArray or vSphere, you can specify the disk when you generate the Portworx spec.
After you apply the Portworx StorageCluster spec, these disks will be created in your environment.
-
From the Distribution Name dropdown menu, select None.
-
(Optional) To customize the configuration options and generate a custom specification, click Customize and perform the following steps:
noteTo continue without customizing the default configuration or generating a custom specification, proceed to Step 7.
- Basic tab:
- To use an existing etcd cluster, do the following:
- Select the Your etcd details option.
- In the field provided, enter the host name or IP and port number.
For example,http://test.com.net:1234
. - Select one of the following authentication methods:
- Disable HTTPS – To use HTTP for etcd communication.
- Certificate Auth – To use HTTPS with an SSL certificate.
For more information, see Secure your etcd communication. - Password Auth – To use HTTPS with username and password authentication.
- To use an internal Portworx-managed key-value store (kvdb), do the following:
- Select the Built-in option.
- To enable TLS encrypted communication among KVDB nodes and between Portworx nodes and the KVDB cluster, select the Enable TLS for internal kvdb checkbox.
- If your cluster does not already have a cert-manager, select the Deploy Cert-Manager for TLS certificates checkbox.
- Select Next.
- To use an existing etcd cluster, do the following:
- Storage tab:
- To enable Portworx to provision drives using a specification, do the following:
- Select the Create Using a Spec option.
- Select one of the following:
- PX-StoreV1 – To designate PX-Store V1 as the datastore.
- PX-StoreV2 – To designate PX-Store V2 as the datastore.
- To add one or more cloud storage drive types for Portworx to use, click + Add Drive and select one of the following types of drives:
- Standard HDD
- Standard SSD
- Premium SSD
- Premium SSD v2
- Ultra disk
noteThe system automatically selects the minimum number of drives to ensure optimal performance.
- Configure the following fields for the drive:
- Select number of VPUs - From the dropdown menu select the number of VPUs for the Oracle block volume.
- Size (GB) - Specify the size of the drive in gigabytes.
- Encryption - Select whether to enable encryption. Options may include None or provider-managed keys.
- Encryption Key - If encryption is enabled, specify the key ID or URI to use.
- Drive Tags - Add labels in key:value format to organize and identify drives. Useful for policies and workload mapping.
- Action - Use the trash icon to remove a drive type from the configuration.
- Max storage nodes per availability zone (Optional): Enter the maximum number of storage nodes that can exist within a single availability zone (failure domain) in your cluster.
- From the Default IO Profile dropdown menu, select Auto.
This enables Portworx to automatically choose the best I/O profile based on detected workload patterns. - From the Journal Device dropdown menu, select one of the following:
- None – To use the default journaling setting.
- Auto – To automatically allocate journal devices.
- Custom – To manually enter a journal device path.
Enter the path of the journal device in the Journal Device Path field.
- To enable Portworx to use all available, unused, and unmounted drives on the node, do the following:
- Select the Consume Unused option.
- From the Journal Device dropdown menu, select one of the following:
- None – To use the default journaling setting.
- Auto – To automatically allocate journal devices.
- Custom – To manually enter a journal device path.
Enter the path of the journal device in the Journal Device Path field.
- Select the PX-StoreV2 checkbox to enable the PX-StoreV2 datastore.
- If you select the PX-StoreV2 checkbox, in the Metadata Path field, enter a pre-provisioned path for storing the Portworx metadata.
The path must be at least 64 GB in size. - Select the Use unmounted disks even if they have a partition or filesystem on it. Portworx will never use a drive or partition that is mounted checkbox to use unmounted disks, even if they contain a partition or filesystem.
Portworx will not use any mounted drive or partition.
- To enable Portworx to use existing drives on a node, do the following:
- Select the Use Existing Drives option.
- In the Drive/Device field, specify the block drive(s) that Portworx uses for data storage.
- In the Pool Label field, assign a custom label in key:value format to identify and categorize storage pools.
- Select the PX-StoreV2 checkbox to enable the PX-StoreV2 datastore.
- If you select the PX-StoreV2 checkbox, in the Metadata Path field, enter a pre-provisioned path for storing the Portworx metadata.
The path must be at least 64 GB in size. - From the Journal Device dropdown menu, select one of the following:
- None – To use the default journaling setting.
- Auto – To automatically allocate journal devices.
- Custom – To manually enter a journal device path.
Enter the path of the journal device in the Journal Device Path field.
- Select Next.
- To enable Portworx to provision drives using a specification, do the following:
- Network tab:
- In the Interface(s) section, do the following:
- Enter the Data Network Interface to be used for data traffic.
- Enter the Management Network Interface to be used for management traffic.
- In the Advanced Settings section, do the following:
- Enter the Starting port for Portworx services.
- Select Next.
- In the Interface(s) section, do the following:
- Customize tab:
- Choose the Kubernetes platform in the Customize section.
- In the Environment Variables section, enter name-value pairs in the respective fields.
- In the Registry and Image Settings section:
- Enter the Custom Container Registry Location to download the Docker images.
- Enter the Kubernetes Docker Registry Secret that serves as the authentication to access the custom container registry.
- From the Image Pull Policy dropdown menu, select Default, Always, IfNotPresent, or Never.
This policy influences how images are managed on the node and when updates are applied.
- In the Security Settings section, select the Enable Authorization checkbox to enable Role-Based Access Control (RBAC) and secure access to storage resources in your cluster.
- In the Advanced Settings section:
- Select the Enable Stork checkbox to enable Stork.
- Select the Enable CSI checkbox to enable CSI.
- Select the Enable Monitoring checkbox to enable monitoring for user-defined projects before installing Portworx Operator.
- Select the Enable Telemetry checkbox to enable telemetry in the StorageCluster spec.
For more information, see Enable Pure1 integration for upgrades on an MKE cluster. - Enter the prefix for the Portworx cluster name in the Cluster Name Prefix field.
- Select the Secrets Store Type from the dropdown menu to store and manage secure information for features such as CloudSnaps and Encryption.
- Click Finish.
- In the summary page, enter a name for the specification in the Spec Name field, and tags in the Spec Tags field.
- Click Download .yaml to download the yaml file with the customized specification or Save Spec to save the specification.
- Click Save & Download to generate the specification.
Deploy Portworx Operator
Use the Operator specifications you generated in the Generate Portworx Specification section, and deploy Portworx Operator by running the following command.
kubectl apply -f 'https://install.portworx.com/<version-number>?comp=pxoperator'
serviceaccount/portworx-operator created
podsecuritypolicy.policy/px-operator created
clusterrole.rbac.authorization.k8s.io/portworx-operator created
clusterrolebinding.rbac.authorization.k8s.io/portworx-operator created
deployment.apps/portworx-operator created
Verify that the Operator is running.
kubectl get pods -lname=portworx-operator -A
kube-system portworx-operator-584ccd74c7-7bhkt 1/1 Running 0 3m48s
Deploy StorageCluster
Use the StorageCluster specifications you generated in the Generate Portworx Specification section, and deploy StorageCluster by running the following command.
kubectl apply -f 'https://install.portworx.com/<version-number>?operator=true&mc=false&kbver=&b=true&c=px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-8dfd338e915b&stork=true&csi=true&mon=true&tel=false&st=k8s&promop=true'
storagecluster.core.libopenstorage.org/px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-8dfd338e915b created
Monitor Portworx nodes
-
Enter the following
kubectl get
command and wait until all Portworx nodes show asReady
orOnline
in the output:kubectl -n <px-namespace> get storagenodes -l name=portworx
NAME ID STATUS VERSION AGE
username-k8s1-node0 xxxxxxxx-xxxx-xxxx-xxxx-43cf085e764e Online 2.11.1-3a5f406 4m52s
username-k8s1-node1 xxxxxxxx-xxxx-xxxx-xxxx-4597de6fdd32 Online 2.11.1-3a5f406 4m52s
username-k8s1-node2 xxxxxxxx-xxxx-xxxx-xxxx-e2169ffa111c Online 2.11.1-3a5f406 4m52s -
Enter the following
kubectl describe
command with theNAME
of one of the Portworx nodes you retrieved in Step 1 to show the current installation status for individual nodes:kubectl -n <px-namespace> describe storagenode <portworx-node-name>
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal PortworxMonitorImagePullInPrgress 7m48s portworx, k8s-node-2 Portworx image portworx/px-enterprise:2.10.1.1 pull and extraction in progress
Warning NodeStateChange 5m26s portworx, k8s-node-2 Node is not in quorum. Waiting to connect to peer nodes on port 9002.
Normal NodeStartSuccess 5m7s portworx, k8s-node-2 PX is ready on this nodenote- In your output, the image pulled will differ based on your chosen Portworx license type and version.
- For Portworx Enterprise, the default license activated on the cluster is a 30 day trial that you can convert to a SaaS-based model or a generic fixed license.
Verify Portworx Pod Status
Enter the following command to list and filter the results for Portworx pods and specify the namespace where you have deployed Portworx:
kubectl get pods -n <px-namespace> -o wide | grep -e portworx -e px
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
portworx-api-774c2 1/1 Running 0 2m55s 192.168.121.196 username-k8s1-node0 <none> <none>
portworx-api-t4lf9 1/1 Running 0 2m55s 192.168.121.99 username-k8s1-node1 <none> <none>
portworx-api-dvw64 1/1 Running 0 2m55s 192.168.121.99 username-k8s1-node2 <none> <none>
portworx-kvdb-94bpk 1/1 Running 0 4s 192.168.121.196 username-k8s1-node0 <none> <none>
portworx-kvdb-8b67l 1/1 Running 0 10s 192.168.121.196 username-k8s1-node1 <none> <none>
portworx-kvdb-fj72p 1/1 Running 0 30s 192.168.121.196 username-k8s1-node2 <none> <none>
portworx-operator-58967ddd6d-kmz6c 1/1 Running 0 4m1s 10.244.1.99 username-k8s1-node0 <none> <none>
prometheus-px-prometheus-0 2/2 Running 0 2m41s 10.244.1.105 username-k8s1-node0 <none> <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-9gs79 2/2 Running 0 2m55s 192.168.121.196 username-k8s1-node0 <none> <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-vpptx 2/2 Running 0 2m55s 192.168.121.99 username-k8s1-node1 <none> <none>
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d-bxmpn 2/2 Running 0 2m55s 192.168.121.191 username-k8s1-node2 <none> <none>
px-csi-ext-868fcb9fc6-54bmc 4/4 Running 0 3m5s 10.244.1.103 username-k8s1-node0 <none> <none>
px-csi-ext-868fcb9fc6-8tk79 4/4 Running 0 3m5s 10.244.1.102 username-k8s1-node2 <none> <none>
px-csi-ext-868fcb9fc6-vbqzk 4/4 Running 0 3m5s 10.244.3.107 username-k8s1-node1 <none> <none>
px-prometheus-operator-59b98b5897-9nwfv 1/1 Running 0 3m3s 10.244.1.104 username-k8s1-node0 <none> <none>
Note the name of a px-cluster
pod. You will run pxctl
commands from these pods in Verify Portworx Cluster Status.
Verify Portworx Cluster Status
You can find the status of the Portworx cluster by running pxctl status
commands from a pod.
Enter the following kubectl exec
command, specifying the pod name you retrieved in Verify Portworx Pod Status:
kubectl exec <pod-name> -n <px-namespace> -- /opt/pwx/bin/pxctl status
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Status: PX is operational
Telemetry: Disabled or Unhealthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e
IP: 192.168.121.99
Local Storage Pool: 1 pool
POOL IO_PRIORITY RAID_LEVEL USABLE USED STATUS ZONE REGION
0 HIGH raid0 3.0 TiB 10 GiB Online default default
Local Storage Devices: 3 devices
Device Path Media Type Size Last-Scan
0:1 /dev/vdb STORAGE_MEDIUM_MAGNETIC 1.0 TiB 14 Jul 22 22:03 UTC
0:2 /dev/vdc STORAGE_MEDIUM_MAGNETIC 1.0 TiB 14 Jul 22 22:03 UTC
0:3 /dev/vdd STORAGE_MEDIUM_MAGNETIC 1.0 TiB 14 Jul 22 22:03 UTC
* Internal kvdb on this node is sharing this storage device /dev/vdc to store its data.
total - 3.0 TiB
Cache Devices:
* No cache devices
Cluster Summary
Cluster ID: px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d
Cluster UUID: xxxxxxxx-xxxx-xxxx-xxxx-6f3fd5522eae
Scheduler: kubernetes
Nodes: 3 node(s) with storage (3 online)
IP ID SchedulerNodeName Auth StorageNode Used Capacity Status StorageStatus Version Kernel OS
192.168.121.196 xxxxxxxx-xxxx-xxxx-xxxx-fad8c65b8edc username-k8s1-node0 Disabled Yes 10 GiB 3.0 TiB Online Up 2.11.0-81faacc 3.10.0-1127.el7.x86_64 CentOS Linux 7 (Core)
192.168.121.99 xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e username-k8s1-node1 Disabled Yes 10 GiB 3.0 TiB Online Up (This node) 2.11.0-81faacc 3.10.0-1127.el7.x86_64 CentOS Linux 7 (Core)
192.168.121.191 xxxxxxxx-xxxx-xxxx-xxxx-19d45b4c541a username-k8s1-node2 Disabled Yes 10 GiB 3.0 TiB Online Up 2.11.0-81faacc 3.10.0-1127.el7.x86_64 CentOS Linux 7 (Core)
Global Storage Pool
Total Used : 30 GiB
Total Capacity : 9.0 TiB
Status displays PX is operational
when the cluster is running as expected. If the cluster is using the PX-StoreV2 datastore, the StorageNode
entries for each node displays Yes(PX-StoreV2)
.
Verify pxctl Cluster Provision Status
-
Access the Portworx CLI.
-
Run the following command to find the storage cluster:
kubectl -n <px-namespace> get storagecluster
NAME CLUSTER UUID STATUS VERSION AGE
px-cluster-xxxxxxxx-xxxx-xxxx-xxxx-3e9bf3cd834d xxxxxxxx-xxxx-xxxx-xxxx-6f3fd5522eae Online 2.11.0 10mThe status must display the cluster is
Online
. -
Run the following command to find the storage nodes:
kubectl -n <px-namespace> get storagenodes
NAME ID STATUS VERSION AGE
username-k8s1-node0 xxxxxxxx-xxxx-xxxx-xxxx-fad8c65b8edc Online 2.11.0-81faacc 11m
username-k8s1-node1 xxxxxxxx-xxxx-xxxx-xxxx-70c31d0f478e Online 2.11.0-81faacc 11m
username-k8s1-node2 xxxxxxxx-xxxx-xxxx-xxxx-19d45b4c541a Online 2.11.0-81faacc 11mThe status must display the nodes are
Online
. -
Verify the Portworx cluster provision status by running the following command.
Specify the pod name you retrieved in Verify Portworx Pod Status.kubectl exec <px-pod> -n <px-namespace> -- /opt/pwx/bin/pxctl cluster provision-status
NODE NODE STATUS POOL POOL STATUS IO_PRIORITY SIZE AVAILABLE USED PROVISIONED ZONE REGION RACK
0c99e1f2-9d49-xxxx-xxxx-xxxxxxxxxxxx Up 0 ( 8ec9e6aa-7726-xxxx-xxxx-xxxxxxxxxxxx ) Online HIGH 32 GiB 32 GiB 33 MiB 0 B default default default
1e89102f-0510-xxxx-xxxx-xxxxxxxxxxxx Up 0 ( 06fcc73a-7e2f-xxxx-xxxx-xxxxxxxxxxxx ) Online HIGH 32 GiB 32 GiB 33 MiB 0 B default default default
24508311-e2fe-xxxx-xxxx-xxxxxxxxxxxx Up 0 ( 58ab2e3f-a22e-xxxx-xxxx-xxxxxxxxxxxx ) Online HIGH 32 GiB 32 GiB 33 MiB 0 B default default default
What to do next
Create a PVC. For more information, see Create your first PVC.