Install Portworx on AWS EKS using eksctl and the Operator


This article provides instructions for installing Portworx on Elastic Kubernetes Service (EKS) using the Weaveworks eksctl command-line utility.

Prerequisites

Before you can install Portworx on EKS using pxctl, you must meet the following prerequisites:

  • You must have eksctl downloaded and installed on your local computer

Install

Grant Portworx the needed AWS permissions

Portworx creates and attaches EBS volumes. As such, it needs the AWS permissions to do so. Below is a sample policy describing these permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "<stmt-id>",
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:ModifyVolume",
                "ec2:DetachVolume",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DeleteTags",
                "ec2:DeleteVolume",
                "ec2:DescribeTags",
                "ec2:DescribeVolumeAttribute",
                "ec2:DescribeVolumesModifications",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DescribeInstances",
                "autoscaling:DescribeAutoScalingGroups"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

You can provide these permissions to Portworx in one of following ways:

  1. Instance Privileges: Provide above permissions for all the instances in the autoscaling cluster by applying the corresponding IAM role. More info about IAM roles and policies can be found here
  2. Environment Variables: Create a User with the above policy and provide the security credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) to Portworx.

Create a ClusterConfig

The ClusterConfig dictates what resources eksctl requests from EKS for the purposes of running Portworx. Portworx requires a number of default resources and configurations in order to function, but other areas of your configuration will vary based on your needs.

  1. Create a ClusterConfig configuration YAML file, specifying your own configuration options for the following:

    • metadata:
      • name: withe cluster name you desire
      • region: with the region you want your eks service to operate from
      • version: with a supported EKS version
    • managedNodeGroups:
      • storage-nodes.instance: with the instance type appropriate for your workloads
      • storage-nodes.minSize: and storage-nodes.maxSize: with the number of worker nodes. Both values must be the same, and a minimum of 3.
      • storage-nodes.ssh.publicKeyPath: if no path is specified, the default will be id_rsa
      • storage-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step
      • storageless-nodes.instanceType: with the instance type approprirate for your storageless node workloads
      • storageless-nodes.minSize: with the minimum number of storageless nodes that can be active on your cluster at any given time
      • storageless-nodes.maxSize: with the maximum number of storageless allowed on your cluster
      • storageless-nodes.desiredCapacity: with the ideal number of storageless nodes preferred on your cluster
      • storageless-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step
    • availabilityZones: with the availabilty zones applicable to your region
    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
    metadata:
      name: px-eksctl
      region: us-east-1
      version: "1.14"
    managedNodeGroups:
      - name: storage-nodes
        instanceType: m4.xlarge
        minSize: 3
        maxSize: 3
        volumeSize: 20
        #ami: auto
        amiFamily: AmazonLinux2
        labels: {role: worker, "portworx.io/node-type": "storage"}
        tags:
          nodegroup-role: worker
        ssh:  
          allow: true
          publicKeyPath: ~/.ssh/aws-vm.pub
        iam:
          attachPolicyARNs:
            - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
            - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
            - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
            - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
            - <arn-of-your-portworx-aws-iam-policy>
          withAddonPolicies:
            imageBuilder: true
            autoScaler: true
            ebs: true
            fsx: true
            efs: true
            albIngress: true
            cloudWatch: true
      - name: storageless-nodes
        instanceType: m4.xlarge
        minSize: 3
        maxSize: 6
        desiredCapacity: 4
        volumeSize: 20
        amiFamily: AmazonLinux2
        labels: {role: worker}
        tags:
          nodegroup-role: worker-storageless
        ssh:
          allow: true
          publicKeyPath: ~/.ssh/aws-vm.pub
        iam:
          attachPolicyARNs:
            - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
            - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
            - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
            - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
            - <arn-of-your-portworx-aws-iam-policy>
          withAddonPolicies:
            imageBuilder: true
            autoScaler: true
            ebs: true
            fsx: true
            efs: true
            albIngress: true
            cloudWatch: true
    availabilityZones: [ 'us-east-1a', 'us-east-1b', 'us-east-1c' ]
  2. Enter the following eksctl create cluster command, specifying the name of the clusterConfig file you created in the step above:

    eksctl create cluster -f <my-clusterConfig>.yml

Install the Operator

Enter the following kubectl create command to deploy the operator:

kubectl create -f https://install.portworx.com/?comp=pxoperator

Generate the spec

To install Portworx with Kubernetes, you must first generate Kubernetes manifests that you will deploy in your cluster:

  1. Navigate to PX-Central and log in or create an account
  2. Select Install and Run to open the Spec Generator

    Screenshot showing install and run

  3. Select New Spec

    Screenshot showing new spec button

  4. Generate a spec with the following selections:

    • On the Storage tab, specify AWS and configure your storage devices based on your needs
    • On the Customize tab, select the Amazon Elastic Container Service for Kubernetes (EKS) option
    • Under the Environment Variables dropdown on the Customize tab, create an environment variable named ENABLE_ASG_STORAGE_PARTITIONING with a value of true

Apply the specs

Apply the generated specs to your cluster.

kubectl apply -f px-spec.yaml

Monitor the Portworx nodes

  1. Enter the following kubectl get command, waiting until all Portworx nodes show as ready in the output:

    kubectl -n kube-system get storagenodes -l name=portworx
  2. Enter the following kubectl describe command with the NAME of one of the Portworx nodes to show the current installation status for individual nodes:

    kubectl -n kube-system describe storagenode <portworx-node-name>
    Events:
    Type     Reason                             Age                     From                  Message
    ----     ------                             ----                    ----                  -------
    Normal   PortworxMonitorImagePullInPrgress  7m48s                   portworx, k8s-node-2  Portworx image portworx/px-enterprise:2.5.0 pull and extraction in progress
    Warning  NodeStateChange                    5m26s                   portworx, k8s-node-2  Node is not in quorum. Waiting to connect to peer nodes on port 9002.
    Normal   NodeStartSuccess                   5m7s                    portworx, k8s-node-2  PX is ready on this node
    NOTE: In your output, the image pulled will differ based on your chosen Portworx license type and version.

Post-Install

Once you have a running Portworx installation, below sections are useful.

Further reading

  • Refer to the eksctl github for more examples of config files which can be used as input to eksctl
  • For more information on what eksctl is, as well as how it works, refer to the eksctl documentation


Last edited: Thursday, Sep 24, 2020