Install Portworx on AWS EKS using eksctl and the Operator

This article provides instructions for installing Portworx on Elastic Kubernetes Service (EKS) using the Weaveworks eksctl command-line utility.

NOTE: You can follow these procedures to deploy Portworx on AWS Outposts.

Prerequisites

Before you can install Portworx on EKS using pxctl, you must meet the following prerequisites:

You must have eksctl downloaded and installed on your local computer

Install

Grant Portworx the needed AWS permissions

Portworx creates and attaches EBS volumes. As such, it needs the AWS permissions to do so. Below is a sample policy describing these permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "<stmt-id>",
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:ModifyVolume",
                "ec2:DetachVolume",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DeleteTags",
                "ec2:DeleteVolume",
                "ec2:DescribeTags",
                "ec2:DescribeVolumeAttribute",
                "ec2:DescribeVolumesModifications",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DescribeInstances",
                "autoscaling:DescribeAutoScalingGroups"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

You can provide these permissions to Portworx in one of the following ways:

Instance Privileges: Provide above permissions for all the instances in the autoscaling cluster by applying the corresponding IAM role. More info about IAM roles and policies can be found here
Environment Variables: Create a User with the above policy and provide the security credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) to Portworx.

Create a ClusterConfig

The ClusterConfig dictates what resources eksctl requests from EKS for the purposes of running Portworx. Portworx requires a number of default resources and configurations in order to function, but other areas of your configuration will vary based on your needs.

Create a ClusterConfig configuration YAML file, specifying your own configuration options for the following:

metadata:
- name: withe cluster name you desire
- region: with the region you want your eks service to operate from
- version: with a supported EKS version
managedNodeGroups:
- storage-nodes.instance: with the instance type appropriate for your workloads
- storage-nodes.minSize: and storage-nodes.maxSize: with the number of worker nodes. Both values must be the same, and a minimum of 3.
- storage-nodes.ssh.publicKeyPath: if no path is specified, the default will be id_rsa
- storage-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step
- storageless-nodes.instanceType: with the instance type approprirate for your storageless node workloads
- storageless-nodes.minSize: with the minimum number of storageless nodes that can be active on your cluster at any given time
- storageless-nodes.maxSize: with the maximum number of storageless allowed on your cluster
- storageless-nodes.desiredCapacity: with the ideal number of storageless nodes preferred on your cluster
- storageless-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step

availabilityZones: with the availability zones applicable to your region

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: px-eksctl
  region: us-east-1
  version: "1.14"
managedNodeGroups:
  - name: storage-nodes
    instanceType: m4.xlarge
    minSize: 3
    maxSize: 3
    volumeSize: 20
    #ami: auto
    amiFamily: AmazonLinux2
    labels: {role: worker, "portworx.io/node-type": "storage"}
    tags:
      nodegroup-role: worker
    ssh:  
      allow: true
      publicKeyPath: ~/.ssh/aws-vm.pub
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
        - <arn-of-your-portworx-aws-iam-policy>
      withAddonPolicies:
        imageBuilder: true
        autoScaler: true
        ebs: true
        fsx: true
        efs: true
        albIngress: true
        cloudWatch: true
  - name: storageless-nodes
    instanceType: m4.xlarge
    minSize: 3
    maxSize: 6
    desiredCapacity: 4
    volumeSize: 20
    amiFamily: AmazonLinux2
    labels: {role: worker}
    tags:
      nodegroup-role: worker-storageless
    ssh:
      allow: true
      publicKeyPath: ~/.ssh/aws-vm.pub
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
        - <arn-of-your-portworx-aws-iam-policy>
      withAddonPolicies:
        imageBuilder: true
        autoScaler: true
        ebs: true
        fsx: true
        efs: true
        albIngress: true
        cloudWatch: true
availabilityZones: [ 'us-east-1a', 'us-east-1b', 'us-east-1c' ]

Enter the following eksctl create cluster command, specifying the name of the clusterConfig file you created in the step above:
```
eksctl create cluster -f <my-clusterConfig>.yml
```

Install the Operator

Enter the following kubectl create command to deploy the operator:

kubectl create -f https://install.portworx.com/?comp=pxoperator

Generate the spec

To install Portworx with Kubernetes, you must first generate Kubernetes manifests that you will deploy in your cluster:

Navigate to PX-Central and log in or create an account
Click Continue with Portworx Enterprise option:
Choose an appropriate license for your requirement and click Continue:
Generate a spec with the following selections:
- On the Storage tab, specify AWS and configure your storage devices based on your needs
- On the Customize tab, select the Amazon Elastic Container Service for Kubernetes (EKS) option
- Under the Environment Variables dropdown on the Customize tab, create an environment variable named ENABLE_ASG_STORAGE_PARTITIONING with a value of true

Apply the specs

Apply the generated specs to your cluster.

kubectl apply -f px-spec.yaml

Monitor the Portworx nodes

Enter the following kubectl get command and wait until all Portworx nodes show as ready in the output:
```
kubectl -n kube-system get storagenodes -l name=portworx
```

Enter the following kubectl describe command with the name of one of the Portworx nodes to show the current installation status for individual nodes:

kubectl -n kube-system describe storagenode <portworx-node-name>

Events:
Type     Reason                             Age                     From                  Message
----     ------                             ----                    ----                  -------
Normal   PortworxMonitorImagePullInPrgress  7m48s                   portworx, k8s-node-2  Portworx image portworx/px-enterprise:2.5.0 pull and extraction in progress
Warning  NodeStateChange                    5m26s                   portworx, k8s-node-2  Node is not in quorum. Waiting to connect to peer nodes on port 9002.
Normal   NodeStartSuccess                   5m7s                    portworx, k8s-node-2  PX is ready on this node

NOTE: In your output, the image pulled will differ based on your chosen Portworx license type and version.

Post-Install

Once you have a running Portworx installation, below sections are useful.