Disk Provisioning on Google Cloud Platform (GCP)


If you are running on GKE, visit Portworx on GKE.

The steps below will help you enable dynamic provisioning of Portworx volumes in your GCP cluster.

Prerequisites

Key-value store

Portworx uses a key-value store for it’s clustering metadata. Please have a clustered key-value database (etcd or consul) installed and ready. For etcd installation instructions refer this doc.

Firewall

Ensure ports 9001-9015 are open between the nodes that will run Portworx. Your nodes should also be able to reach the port KVDB is running on (for example etcd usually runs on port 2379).

Create a GCP cluster

To manage and auto provision GCP disks, Portworx needs access to the GCP Compute Engine API. There are two ways to do this.

Using instance priviledges

Give your instances priviledges to access the GCP API server. This is the preferred method since it requires the least amount of setup on each instance.

Using an account file

Alternatively, you can give Portworx access to the GCP API server via an account file and environment variables. First, you will need to create a service account in GCP and download the account file.

  1. Create a service account in the “Service Account” section that has the Compute Engine admin role.
  2. Go to IAM & admin -> Service Accounts -> (Instance Service Account) -> Select “Create Key” and download the .json file

This json file needs to be made available on any GCP instance that will run Portworx. Place this file under a /etc/pwx/ directory on each GCP instance. For example, /etc/pwx/gcp.json.

Install

If you used an account file above, you will have to configure the PX installation arguments to access this file by way of it’s environmnet variables. In the installation arguments for PX, pass in the location of this file via the environment variable GOOGLE_APPLICATION_CREDENTIALS. For example, use -e GOOGLE_APPLICATION_CREDENTIALS=/etc/pwx/gcp.json.

If you installing on Kuberenetes, you can use a secret to mount /etc/pwx/gcp.json into the Portworx Daemonset and then expose GOOGLE_APPLICATION_CREDENTIALS as an env in the Daemonset.

Follow these instructions to install Portworx based on your container orchestration environment.

Disk template

Portworx takes in a disk spec which gets used to provision GCP persistent disks dynamically.

A GCP disk template defines the Google persistent disk properties that Portworx will use as a reference. There are 2 ways you can provide this template to Portworx.

1. Using a template specification

The spec follows the following format:

"type=<GCP disk type>,size=<size of disk>"
  • type: Following two types are supported
    • pd-standard
    • pd-ssd
  • size: This is the size of the disk in GB

See GCP disk for more details on above parameters.

Examples:

  • "type=pd-ssd,size=200"
  • "type=pd-standard,size=200", "type=pd-ssd,size=100"

2. Using existing GCP disks as templates

You can also reference an existing GCP disk as a template. On every node where PX is brought up as a storage node, a new GCP disk(s) identical to the template will be created.

For example, if you created a template GCP disk called px-disk-template-1, you can pass this in to PX as a parameter as a storage device.

Ensure that these disks are created in the same zone as the GCP node group.

Limiting storage nodes

PX allows you to create a heterogenous cluster where some of the nodes are storage nodes and rest of them are storageless.

You can specify the number of storage nodes in your cluster by setting the max_storage_nodes_per_zone input argument. This instructs PX to limit the number of storage nodes in one zone to the value specified in max_storage_nodes_per_zone argument. The total number of storage nodes in your cluster will be

Total Storage Nodes = (Num of Zones) * max_storage_nodes_per_zone.

While planning capacity for your auto scaling cluster make sure the minimum size of your cluster is equal to the total number of storage nodes in PX. This ensures that when you scale up your cluster, only storage less nodes will be added. While when you scale down the cluster, it will scale to the minimum size which ensures that all PX storage nodes are online and available.

You can always ignore the max_storage_nodes_per_zone argument. When you scale up the cluster, the new nodes will also be storage nodes but while scaling down you will loose storage nodes causing PX to loose quorum.

Examples:

  • "-s", "type=gp2,size=200", "-max_storage_nodes_per_zone", "1"

For a cluster of 6 nodes spanning 3 zones (us-east-1a,us-east-1b,us-east-1c), in the above example PX will have 3 storage nodes (one in each zone) and 3 storage less nodes. PX will create a total 3 disks of size 200 each and attach one disk to each storage node.

  • "-s", "type=gp2,size=200", "-s", "type=io1,size=100,iops=1000", "-max_storage_nodes_per_zone", "2"

For a cluster of 9 nodes spanning 2 zones (us-east-1a,us-east-1b), in the above example PX will have 4 storage nodes and 5 storage less nodes. PX will create a total of 8 disks (4 of size 200 and 4 of size 100). PX will attach a set of 2 disks (one of size 200 and one of size 100) to each of the 4 storage nodes.


Last edited: Tuesday, Dec 4, 2018