This guide shows you how you can easily deploy Portworx on Amazon Elastic Container Service ECS

Step 1: Create an ECS cluster

In this example, we create an ECS cluster called ecs-demo1 using default AWS AMI (ami-b2df2ca4) and create two EC2 instances in the US-EAST-1 region.

As of this guide is written, the default ECS AMI uses Docker 1.12.6. Note that Portworx recommends a minimum cluster size of 3 nodes.

Create the cluster in the console

Log into the ECS console and create an ecs cluster called “ecs-demo1”.

ecs-clust-create.

On above, the Container Instance IAM role is used by the ECS container agent. These ECS container agent is deployed by default with the EC2 instances from the ECS wizard. And these agent makes call to AWS ECS API actions on your behalf, thus these EC2 instances that running the ECS container agent require an IAM role that has permission to join ECS cluster and launch containers within the cluster.

Create a custom IAM role and Select Role Type “Amazon EC2 Role for Container Service”. This is the minimal required permission to launch ECS cluster. And depends on your use case, you may need additional AWS policy for your ECS to access and use other AWS resources. The “AmazonEC2ContainerServiceRole” has the policy shown below:

{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:Describe*",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:DeregisterTargets",
        "elasticloadbalancing:Describe*",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
        "elasticloadbalancing:RegisterTargets"
         ],
        "Resource": "*"
      }
      ]
 }

Use the created custom IAM role ECS for this ECS cluster and the security group should allow inbound ssh access from your network.

Your EC2 instances must have the correct IAM role set. Follow these IAM instructions.

After the ECS cluster “ecs-demo1” successfully launched, the corresponding EC2 instances that belong to this ECS cluster can be found under the “ECS instance” tab of ECS console or from AWS EC2 console. Each of this EC2 instance is running with an amazon-ecs-agent in docker container.

ecs-clust-create.

Add storage capacity to each EC2 instance

Provisioning storage to these EC2 instances by creating new EBS volumes and attaching them to these EC2 instances. Portworx will be using these EBS volumes to provision storage to your containers. Below we are creating a 20GB EBS volume on the same region “us-east-1b” of the launched EC2 instances. Ensure all ECS instances are attached with the EBS volumes.

ecs-clust-create.

ecs-clust-create.

Note that there is no need to format the EBS volumes once they are created and attached to the ec2 instance. Portworx will pick up the available unformatted drives (if you use the -a option as show below in the next step) or you can point to the appropriate block device for the Portworx to pick up by using the -s option when you launch Portworx with the docker run command.

Step 2: Deploy Portworx

Install PX on each ECS instance. Portworx will use the EBS volumes you provisioned in step 4.

The installation and setup of PX OCI bundle is a 4-step process:

  1. Install the PX OCI bundle
  2. Configure PX under runC
  3. Download and Activate the Portworx service
  4. Enable log rotation

Step 2.1: Install the PX OCI bundle

Portworx provides a Docker based installation utility to help deploy the PX OCI bundle. This bundle can be installed by running the following Docker container on your host system:

latest_stable=$(curl -fsSL 'https://install.portworx.com?type=dock&stork=false' | awk '/image: / {print $2}')
# Download OCI bits (reminder, you will still need to run `px-runc install ..` after this step)
sudo docker run --entrypoint /runc-entry-point.sh \
    --rm -i --privileged=true \
    -v /opt/pwx:/opt/pwx -v /etc/pwx:/etc/pwx \
    $latest_stable

Step 2.2: Configure PX under runC

Now that the PX OCI bundle has been deployed, we have to configure it by running the following:

# Basic installation
sudo /opt/pwx/bin/px-runc install -sysd /dev/null -c MY_CLUSTER_ID \
  -k etcd://myetc.company.com:2379 \
  -s /dev/xvdb -s /dev/xvdc 
Command-line arguments

Options

-c                        [REQUIRED] Specifies the cluster ID that this PX instance is to join
-k                        [REQUIRED] Points to your key value database, such as an etcd cluster or a consul cluster
-s                        [REQUIRED unless -a is used] Specifies the various drives that PX should use for storing the data
-e key=value              [OPTIONAL] Specify extra environment variables
-v <dir:dir[:shared,ro]>  [OPTIONAL] Specify extra mounts
-d <ethX>                 [OPTIONAL] Specify the data network interface
-m <ethX>                 [OPTIONAL] Specify the management network interface
-z                        [OPTIONAL] Instructs PX to run in zero storage mode
-f                        [OPTIONAL] Instructs PX to use an unmounted drive even if it has a filesystem on it
-a                        [OPTIONAL] Instructs PX to use any available, unused and unmounted drives
-A                        [OPTIONAL] Instructs PX to use any available, unused and unmounted drives or partitions
-j                        [OPTIONAL] Specifies a journal device for PX.  Specify a persistent drive like /dev/sdc or use auto (recommended)
-x <swarm|kubernetes>     [OPTIONAL] Specify scheduler being used in the environment
-r <portnumber>           [OPTIONAL] Specifies the portnumber from which PX will start consuming. Ex: 9001 means 9001-9020
  • additional PX-OCI -specific options:
-oci <dir>                [OPTIONAL] Specify OCI directory (default: /opt/pwx/oci)
-sysd <file>              [OPTIONAL] Specify SystemD service file (default: /etc/systemd/system/portworx.service)

KVDB options

-userpwd <user:passwd>    [OPTIONAL] Username and password for ETCD authentication
-ca <file>                [OPTIONAL] Specify location of CA file for ETCD authentication
-cert <file>              [OPTIONAL] Specify location of certificate for ETCD authentication
-key <file>               [OPTIONAL] Specify location of certificate key for ETCD authentication
-acltoken <token>         [OPTIONAL] ACL token value used for Consul authentication

Secrets options

-secret_type <aws|kvdb|vault>   [OPTIONAL] Specify the secret type to be used by Portworx for cloudsnap and encryption features.
-cluster_secret_key <id>        [OPTIONAL] Specify the cluster wide secret key to be used when using AWS KMS or Vault for volume encryption.

Environment variables

PX_HTTP_PROXY         [OPTIONAL] If running behind an HTTP proxy, set the PX_HTTP_PROXY variables to your HTTP proxy.
PX_HTTPS_PROXY        [OPTIONAL] If running behind an HTTPS proxy, set the PX_HTTPS_PROXY variables to your HTTPS proxy.
PX_ENABLE_CACHE_FLUSH [OPTIONAL] Enable cache flush deamon. Set PX_ENABLE_CACHE_FLUSH=true.
PX_ENABLE_NFS         [OPTIONAL] Enable the PX NFS daemon. Set PX_ENABLE_NFS=true.

NOTE: Setting environment variables can be done using the -e option, during PX-OCI or PX Docker Container command line install (e.g. add -e VAR=VALUE option).

# Example PX-OCI config with extra "PX_ENABLE_CACHE_FLUSH" environment variable
$ sudo /opt/pwx/bin/px-runc install -e PX_ENABLE_CACHE_FLUSH=yes \
    -c MY_CLUSTER_ID -k etcd://myetc.company.com:2379 -s /dev/xvdb

Step 2.3: Download and Activate the Portworx service

Since the Amazon ECS systems do not have the systemd service available, we will need to start Portworx service via the custom init-script:

sudo curl https://docs.portworx.com/cloud/aws/portworx-sysvinit.sh -o /etc/rc.d/init.d/portworx
sudo chmod 755 /etc/rc.d/init.d/portworx
sudo chkconfig --add portworx
sudo service portworx start

Step 2.4: Enable log rotation

Finally, since the Portworx service creates some amount of log-files, we need to ensure these logs are recycled on regular basis, using systems’ “logrotate” service:

cat > /etc/logrotate.d/portworx << _EOF
/var/log/portworx.log {
  minsize 50M
  daily
  rotate 5
  missingok
  compress
  notifempty
  nocreate
  postrotate
      service portworx restart >/dev/null 2>&1 || true
  endscript
}
_EOF

Step 3: Setup ECS task with PX volume from ECS CLI workstation

From your linux workstation download and setup AWS ECS CLI utilities

  1. Download and install ECS CLI (detail instructions)

    $ sudo curl -o /usr/local/bin/ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest
    $ sudo chmod +x /usr/local/bin/ecs-cli
    
  2. Configure AWS ECS CLI on your workstation

    $ export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXX
    $ export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXX
    $ ecs-cli configure --region us-east-1 --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --cluster ecs-demo1
    
  3. Create a 1GB PX volume using the Docker CLI. ssh into one of the ECS instances and create this PX volumes.

    $ ssh -i ~/.ssh/id_rsa ec2-user@52.91.191.220
    $ docker volume create -d pxd --name=demovol --opt size=1 --opt repl=3 --opt shared=true
    demovol
    
    $ docker volume ls
    DRIVER              VOLUME NAME
    pxd                 demovol
    
  4. From your ECS CLI workstation which has ecs-cli command; setup and launch ECS task definition with previously created PX volume. Create a task definition file “redis.yml” which will launch two containers: redis based on redis image, and web based on binocarlos/moby-counter. Then use ecs-cli command to post this task definition and launch it.

    $ cat redis.yml
    web:
    image: binocarlos/moby-counter
    links:
      -  redis:redis
    redis:
    image: redis
    volumes:
      -  demovol:/data
      -
    $ ecs-cli compose --file redis.yml up
    INFO[0001] Using ECS task definition                     TaskDefinition="ecscompose-root:1"
    INFO[0001] Starting container...                         container="59701c44-c267-4c85-a8c0-ff87910af535/web"
    INFO[0001] Starting container...                         container="59701c44-c267-4c85-a8c0-ff87910af535/redis"
    INFO[0001] Describe ECS container status                 container="59701c44-c267-4c85-a8c0-ff87910af535/redis" desiredStatus=RUNNING lastStatus=PENDING taskDefinition="ecscompose-root:1"
    INFO[0001] Describe ECS container status                 container="59701c44-c267-4c85-a8c0-ff87910af535/web" desiredStatus=RUNNING lastStatus=PENDING taskDefinition="ecscompose-root:1"
    INFO[0013] Started container...                          container="59701c44-c267-4c85-a8c0-ff87910af535/redis" desiredStatus=RUNNING lastStatus=RUNNING taskDefinition="ecscompose-root:1"
    INFO[0013] Started container...                          container="59701c44-c267-4c85-a8c0-ff87910af535/web" desiredStatus=RUNNING lastStatus=RUNNING taskDefinition="ecscompose-root:1"
    
    $ ecs-cli ps
    Name                                               State    Ports                                                          TaskDefinition
    59701c44-c267-4c85-a8c0-ff87910af535/redis         RUNNING                                                                 ecscompose-root:1
    59701c44-c267-4c85-a8c0-ff87910af535/web           RUNNING                                                                 ecscompose-root:1
    
  5. You can also view the task status in the ECS console.

task

  1. On the above ECS console, Clusters -> pick your cluster ecs-demo1 and click on the Container Instance ID that corresponding to the running task. This will display the containers information including where are these containers deployed into which EC2 instance. Below, we find that the task defined containers are deployed on EC2 instance with public IP address 52.91.191.220. task
  2. From above, ssh into the EC2 instance 52.91.191.220 and verify PX volume is attached to running container.

    [ec2-user@ip-172-31-31-61 ~]$ sudo docker ps -a
    CONTAINER ID        IMAGE                            COMMAND                  CREATED             STATUS              PORTS                                             NAMES
    7ba93d51918b        binocarlos/moby-counter          "node index.js"          12 hours ago        Up 12 hours         80/tcp                                            ecs-ecscompose-root-1-web-c2fbfff3bf92b1dad401
    e25ba9131f9b        redis                            "docker-entrypoint.sh"   12 hours ago        Up 12 hours         6379/tcp                                          ecs-ecscompose-root-1-redis-a6a6a2fcb4a6d188e601
    
    [ec2-user@ip-172-31-31-61 ~]$ sudo /opt/pwx/bin/pxctl v l
    ID                      NAME                    SIZE    HA      SHARED  ENCRYPTED       IO_PRIORITY     SCALE   STATUS
    1061916907972944739     demovol                 1 GiB   3       yes     no              LOW             0       up - attached on 172.31.31.61
    
  3. Check the redis container ecs-ecscompose-root-1-redis-a6a6a2fcb4a6d188e601 and verify a 1GB pxfs volume is mounted on /data

    [ec2-user@ip-172-31-31-61 ~]$ sudo docker exec -it ecs-ecscompose-root-1-redis-a6a6a2fcb4a6d188e601 sh -c 'df -kh'
    Filesystem                                                                                        Size  Used Avail Use% Mounted on
    /dev/mapper/docker-202:1-263203-3f7e353e23d7ba722fc74d1fb7db60e34f98933355ac65f78e6b4f2bcde19778  9.8G  215M  9.0G   3% /
    tmpfs                                                                                             3.9G     0  3.9G   0% /dev
    tmpfs                                                                                             3.9G     0  3.9G   0% /sys/fs/cgroup
    pxfs                                                                                              976M  2.5M  907M   1% /data
    /dev/xvda1                                                                                        7.8G  1.3G  6.5G  16% /etc/hosts
    shm                                                                                                64M     0   64M   0% /dev/shm
    

Step 4: Setup ECS task with PX volume via AWS ECS console

Optional: the same process of step3 but do it on AWS GUI

Create a ECS tasks definition directly via the ECS console (GUI) and using PX volume.

  1. ssh into one of the EC2 instance and create a new PX volume using Docker CLI.

     $ docker volume create -d pxd --name=demovol --opt size=1 --opt repl=3 --opt shared=true
    
  2. In AWS ECS console, choose the previously created cluster ecs-demo1; then create a new task definition.

task

  1. From the new task definition screen, enter the task definition name redis-demo and click Add volume near the bottom of the page. task
  2. Enter the Name in the Add volume screen, that is just the name for your volume defined in this task definition and no need to be the same as the PX volume name. Then enter the Source path, and this is the PX volume name demovol. task
  3. After added the volume, click Add container button to define your containers specification. task
  4. From the Add container screen, enter the Container name “redis” and Image* “redis” ; then click the Add button. task
  5. Adding another container, on the same Create a Task Definition screen, click Add container button. On the Add container screen, enter the Container name “web” and Image* “binocarlos/moby-counter” and On NETWORK SETTINGS Links enter “redis:redis” ; then on STORAEE AND LOGGING Mount Points select from drop down “volume0” and enter the Container path “/data” ; and then click Add button. task
  6. On the same task definition screen, click create button at the of the screen. task
  7. From the AWS ECS console, Task Definitions, select the definition “redis-demo” and click Actions and select run task
  8. Click Run Task task
  9. You will see the task is submitted and change status from PENDING to RUNNING. task task