k8s porx Logo

This is a guide to setup a production ready Portworx cluster using Kubernetes (KOPS+AWS) environment that allows you to dynamically provision persistent volumes. KOPS helps you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters. Under the hood KOPS uses AWS Autoscaling groups (ASG) to spin up EC2 instances.

Portworx in an Auto Scaling Group

EC2 instances in an ASG are ephemeral in nature. In such an environment Portworx can create EBS volumes based on an input template whenever a new instance spins up and provision persistent volumes for your applications. Portworx fingerprints the EBS volumes and attaches them to an instance in the autoscaling cluster. In this way an ephemeral instance gets its own identity. When an instance terminates, the auto scaling group will automatically add a new instance to the cluster. Portworx gracefully handle this scenario by re-attaching the old EBS volumes to it and give a new instance the old identity. In this way all the data is retained with zero storage downtime.


Key-value store

Portworx uses a key-value store for it’s clustering metadata. Please have a clustered key-value database (etcd or consul) installed and ready. For etcd installation instructions refer this doc.

Shared mounts

Portworx 1.3 and higher automatically enables shared mounts.

If you are installing Portworx 1.2, you must configure Docker to allow shared mounts propagation (see instructions), as otherwise Portworx will fail to start.


Ensure ports 9001-9015 are open between the nodes that will run Portworx. Your nodes should also be able to reach the port KVDB is running on (for example etcd usually runs on port 2379).

In AWS, this can be done through the security group of the VPC to which your instances belong.


Ensure all nodes running PX are time-synchronized, and NTP service is configured and running.

KOPS cluster in AWS

Detailed instructions on how to setup a KOPS cluster in AWS are documented here.

AWS Requirements

As Portworx needs to create and attach EBS volumes, it needs corresponding AWS permissions. Following is a sample policy describing those permissions:

    "Version": "2012-10-17",
    "Statement": [
            "Sid": "<stmt-id>",
            "Effect": "Allow",
            "Action": [
            "Resource": [

You can provide these permissions to Portworx in one of following ways:

  1. Instance Privileges: Provide above permissions for all the instances in the autoscaling cluster by applying the corresponding IAM role. More info about IAM roles and policies can be found here
  2. Environment Variables: Create a User with the above policy and provide the security credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) to Portworx.

EBS volume template

An EBS volume template defines a set of EBS volumes that Portworx will use as a reference. There are 2 ways you can provide this template to Portworx.

Using existing EBS volumes as templates

Create at least one EBS volume using the AWS console or AWS CLI. This volume (or a set of volumes) will serve as a template EBS volume(s). On every node where PX is brought up as a storage node, a new EBS volume(s) identical to the template volume(s) will be created.

For example, create two volumes as:

vol-0743df7bf5657dad8: 1000 GiB provisioned IOPS
vol-0055e5913b79fb49d: 1000 GiB GP2

Ensure that these EBS volumes are created in the same region as the auto scaling group.

Record the EBS volume ID (e.g. vol-04e2283f1925ec9ee), this will be passed in to PX as a parameter as a storage device.

Using a template specification

For PX 1.3 and higher, you can specify a template spec which will be used by Portworx to create new EBS volumes.

The spec follows the following format:

"type=<EBS volume type>,size=<size of EBS volume>,iops=<IOPS value>"
  • type: Following two types are supported
    • gp2
    • io1 (For io1 volumes specifying the iops value is mandatory.)
  • size: This is the size of the EBS volume in GB
  • iops: This is the required IOs per second from the EBS volume.

See EBS details for more details on above parameters.


  • "type=gp2,size=200"
  • "type=gp2,size=100","type=io1,size=200,iops=1000"

We will supply the template(s), when we create the Portworx DaemonSet spec later in this guide.


Portworx gets deployed as a Kubernetes DaemonSet. Following sections describe how to generate the spec files and apply them.

Generate the Portworx Spec

When generating the spec, following parameters are important for KOPS:

  1. AWS environment variables: In the environment variables option (e), specify AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for the KOPS IAM user. Example: AWS_ACCESS_KEY_ID=<id>,AWS_SECRET_ACCESS_KEY=<key>. If you are using instance privileges you can ignore setting the environment variables.

  2. Volume template: In the drives option (s), specify the EBS volume template that you created in previous step. Portworx will dynamically create EBS volumes based on this template.

To generate the spec file for the 1.2 release, head on to 1.2 install page.

To generate the spec file for the 1.3 release, head on to 1.3 install page.

Alternately, you can use curl to generate the spec as described in Generating Portworx Kubernetes spec using curl.

Secure ETCD:
If using secure etcd provide “https” in the URL and make sure all the certificates are in the /etc/pwx/ directory on each host which is bind mounted inside PX container.

Installing behind the HTTP proxy

During the installation Portworx may require access to the Internet, to fetch kernel headers if they are not available locally on the host system. If your cluster runs behind the HTTP proxy, you will need to expose PX_HTTP_PROXY and/or PX_HTTPS_PROXY environment variables to point to your HTTP proxy when starting the DaemonSet.

Use e=PX_HTTP_PROXY=<http-proxy>,PX_HTTPS_PROXY=<https-proxy> query param when generating the DaemonSet spec.

Apply the spec

Once you have generated the spec file, deploy Portworx.

kubect apply -f px-spec.yaml

Monitor the portworx pods

kubectl get pods -o wide -n kube-system -l name=portworx

Monitor Portworx cluster status

PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status

If you are still experiencing issues, please refer to Troubleshooting PX on Kubernetes and General FAQs.

Corelating EBS volumes with Portworx nodes

Portworx when running in ASG mode provides a set of CLI commands to display the information about all EBS volumes and their attachment information.

Note: Following commands are only available for PX version > 1.3

Listing all Cloud Drives

Run the following command to display all the cloud drives being used by Portworx.

# kubectl exec -it $PX_POD /opt/pwx/bin/pxctl clouddrive list

Cloud Drives Summary
        Number of nodes in the cluster:  3
        Number of drive sets in use:  3
        List of storage nodes:  [ip-172-20-52-178.ec2.internal ip-172-20-53-168.ec2.internal ip-172-20-33-108.ec2.internal]
        List of storage less nodes:  []

Drive Set List
        NodeIndex        NodeID                                InstanceID                Zone                Drive IDs
        0                ip-172-20-53-168.ec2.internal        i-0347f50a091716c66        us-east-1a        vol-0a3ff5863c7b2c2e4, vol-0f821f3e3a884e275
        1                ip-172-20-33-108.ec2.internal        i-089b22fc89bb11a92        us-east-1a        vol-048dd9c1fd5ed421d, vol-012a4ed30013590ef
        2                ip-172-20-52-178.ec2.internal        i-09169ceb37b251bac        us-east-1a        vol-0bd9aaab0fb615351, vol-0c9f027d111844227

Inspecting Cloud Drives

Run the following command to display more information about the drives attached on a node.

# kubectl exec -it $PX_POD /opt/pwx/bin/pxctl clouddrive inspect --nodeid ip-172-20-53-168.ec2.internal

Drive Set Configuration
        Number of drives in the Drive Set:  2
        NodeID:  ip-172-20-53-168.ec2.internal
        NodeIndex:  0
        InstanceID:  i-0347f50a091716c66
        Zone:  us-east-1a

        Drive  0
                ID:  vol-0a3ff5863c7b2c2e4
                Type:  io1
                Size:  16 Gi
                Iops:  100
                Path:  /dev/xvdf

        Drive  1
                ID:  vol-0f821f3e3a884e275
                Type:  gp2
                Size:  8 Gi
                Iops:  100
                Path:  /dev/xvdg

Deploy a sample application

Now that you have Portworx installed, checkout various examples of applications using Portworx on Kubernetes.