Run Portworx Developer Edition with Docker


To install and configure Portworx Developer Edition via the Docker CLI, use the command-line steps described in this section.

Important: Portworx stores configuration metadata in a KVDB (key/value store), such as Etcd or Consul. Portworx, Inc. recommends setting up a dedicated kvdb for Portworx to use. If you want to set one up, see the etcd example for Portworx.

Install and configure Docker

Follow the Docker install guide to install and start the Docker Service.

Specify storage

Portworx pools the storage devices on your server and creates a global capacity for containers. The following example uses the two non-root storage devices (/dev/xvdb, /dev/xvdc).

Important: Back up any data on storage devices that will be pooled. Storage devices will be reformatted!

First, use this command to view the storage devices on your server:

lsblk
    NAME                      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
    xvda                      202:0    0     8G  0 disk
    └─xvda1                   202:1    0     8G  0 part /
    xvdb                      202:16   0    64G  0 disk
    xvdc                      202:32   0    64G  0 disk
Note that devices formatted with a partition are shown under the TYPE column as part.

Next, identify the storage devices you will be allocating to Portworx. Portworx can run in a heterogeneous environment, so you can mix and match drives of different types. Different servers in the cluster can also have different drive configurations.

Run Portworx

You can now run Portworx via the Docker CLI as follows:

if `uname -r | grep -i coreos > /dev/null`; \
then HDRS="/lib/modules"; \
else HDRS="/usr/src"; fi
sudo docker run --restart=always --name px -d --net=host       \
                 --privileged=true                             \
                 -v /run/docker/plugins:/run/docker/plugins    \
                 -v /var/lib/osd:/var/lib/osd:shared           \
                 -v /dev:/dev                                  \
                 -v /etc/pwx:/etc/pwx                          \
                 -v /opt/pwx/bin:/export_bin                   \
                 -v /var/run/docker.sock:/var/run/docker.sock  \
                 -v /var/cores:/var/cores                      \
                 -v ${HDRS}:${HDRS}                            \
                portworx/px-dev -k etcd://myetc.company.com:2379 -c MY_CLUSTER_ID -s /dev/sdb -s /dev/sdc

Portworx daemon arguments

The following arguments are provided to the Portworx daemon:

Below is the list of arguments you can pass to px-runc:

General options
-c <id>                   [REQUIRED] Specifies the cluster ID that this PX instance is to join
-k <kvdb://host:port>     [REQUIRED] Points to your key value database, such as an etcd cluster or a consul cluster
-b                        Use in-built kvdb. Provide the kvdb endpoints required for bootstrap with -k option.
-s <device path>          [REQUIRED unless -a/-A are used] Specify storage devices that PX should use for storing the data
-xs <omit device path>    Specify storage devices that PX should NOT use for storing the data (useful with -a/-A)
-T <type>                 Specify backend storage type (<type> is mdraid or lvm)
-cache [<device path>]    Specify storage devices that PX should use for caching
-dedicated_cache          Constrain cache drive assignment from given -cache drives only
-j <device path>          Specify storage device that PX should use for storing the journal data
-metadata <device path>   Specify storage device that PX should use for storing the system meta data
-kvdb_dev <device path>   Specify storage device that PX should use for storing internal kvdb data
-oci <dir>                Specify OCI directory (dfl: /opt/pwx/oci)
-sysd <file>              Specify SystemD service file (dfl: /etc/systemd/system/portworx.service)
-e key=value              Specify extra environment variables
-v <dir:dir[:shared,ro]>  Specify extra mounts
-d <ethX>                 Specify the data network interface
-m <ethX>                 Specify the management network interface
-z                        Instructs PX to run in zero storage mode
-f                        Instructs PX to use an unmounted drive even if it has a filesystem on it
-a                        Instructs PX to use any available, unused and unmounted drives
-A                        Instructs PX to use any available, unused and unmounted drives or partitions
-x <swarm|kubernetes>     Specify scheduler type (if PX running in scheduler environment)
-r <startport>            Start of the portrange Portworx will use for communication (dfl: 9001)
-marketplace_name         [OPTIONAL] pass in the marketplace name if installing via a 3rd party marketplace
KVDB options
-userpwd <user:passwd>    Username and password for ETCD authentication
-ca <file>                Specify location of CA file for ETCD authentication
-cert <file>              Specify location of certificate for ETCD authentication
-key <file>               Specify location of certificate key for ETCD authentication
-acltoken <token>         Specify ACL token for Consul authentication
+internal-kvdb-options:
-kvdb_cluster_size <#>    Size of the internal kvdb cluster (dfl: 3)
-kvdb_recovery            Starts the nodes in kvdb recovery mode
Cluster domain options
-cluster_domain <name>    Cluster Domain Name for this cluster
PX-API options
# px-api-ssl-options:
-apirootca <file>         Specify self-signed root CA certificate file
-apicert <file>           Specify node certificate file
-apikey <file>            Specify node certificate key file
-apidisclientauth         Disable api client authentication
# px-authentication-options:
-oidc_issuer   <URL>          Location of OIDC service (e.g. https://accounts.google.com)
-oidc_client_id <id>          Client id provided by the OIDC
-oidc_custom_claim_namespace  OIDC namespace for custom claims
-jwt_issuer <val>             JSON Web Token issuer (e.g. openstorage.io)
-jwt_rsa_pubkey_file <file>   JSON Web Token RSA Public file path
-jwt_ecds_pubkey_file <file>  JSON Web Token ECDS Public file path
-username_claim <claim>       Claim key from the token to use as the unique id of the user (<claim> is sub, email or name; dfl: sub)
Volume options
-disable-sharedv4         Disable sharedv4 volume support. When set, NFS dependencies will not be installed.
-raid <0|10>              Specify which RAID-level should PX use with local storage (dfl: 0)
-enable-mpcfg-update     Enable Multipath configuration update to blacklist PX devnodes.
The -raid <0|10> option is different than the volume replication factor. For example, Portworx nodes using -raid 10 and hosting volumes with a replication factor of 3, will keep 6 copies of the data.
CSI options
-csiversion <ver>         Specify which CSI version to use (<ver> is 1.0 or 0.3; dfl: 1.0)
secrets options
-secret_type <type>       Specify the secrets type (<type> is aws-kms, dcos, docker, ibm-kp, k8s, kvdb, vault, gcloud-kms or azure-kv)
-cluster_secret_key <id>  Specify cluster-wide secret ID
Auto-scaling group options
-max_drive_set_count <#>         Specify maximum number of drive sets PX can create
-max_storage_nodes_per_zone <#>  Specify the maximum number of storage nodes per zone in PX cluster
-node_pool_label <key>           Specify the scheduler node label key with which nodes are grouped into node pools
Resource control options
--cpus <#.#>                  Specify maximum number of CPUs Portworx can use (e.g. --cpus=1.5)
--cpu-shares <#>              Specify CPU shares (relative weight)
--cpuset-cpus <val>           Specify CPUs in which to allow execution (<val> is range <#-#>, or sequence <#,#>)
--memory <bytes>              Specify maximum ammount of memory Portworx can use
--memory-reservation <bytes>  Specify memory reservation soft limit (must be smaller than '--memory')
--memory-swap <bytes>         Specify maximum ammount of RAM+SWAP memory Portworx can use
--memory-swappiness <0-100>   Specify percentage of container's anonymous pages host can swap out

Environment variables
PX_HTTP_PROXY          If running behind an HTTP proxy, set the PX_HTTP_PROXY variables to your HTTP proxy.
PX_HTTPS_PROXY         If running behind an HTTPS proxy, set the PX_HTTPS_PROXY variables to your HTTPS proxy.
PX_ENABLE_CACHE_FLUSH  To enable cache flush daemon, set PX_ENABLE_CACHE_FLUSH=true.
You can set the environment variables using the -e option.

For example, to set the PX_ENABLE_CACHE_FLUSH environment variable to true, run the following command:

sudo /opt/pwx/bin/px-runc install -e PX_ENABLE_CACHE_FLUSH=yes \
    -c MY_CLUSTER_ID -k etcd://myetc.company.com:2379 -s /dev/xvdb

Docker runtime command options

The relevant Docker runtime command options are explained below:

--privileged
    > Sets Portworx to be a privileged container. Required to export block device and for other functions.

--net=host
    > Sets communication to be on the host IP address over ports 9001 -9003. Future versions will support separate IP addressing for Portworx.

--shm-size=384M
    > Portworx advertises support for asynchronous I/O. It uses shared memory to sync across process restarts

-v /run/docker/plugins
    > Specifies that the volume driver interface is enabled.

-v /dev
    > Specifies which host drives Portworx can access. Note that Portworx only uses drives specified in config.json. This volume flag is an alternate to --device=\[\].

-v /etc/pwx/config.json:/etc/pwx/config.json
    > the configuration file location.

-v /var/run/docker.sock
    > Used by Docker to export volume container mappings.

-v /var/lib/osd:/var/lib/osd:shared
    > Location of the exported container mounts. This must be a shared mount.

-v /opt/pwx/bin:/export_bin
    > Exports the Portworx command line (**pxctl**) tool from the container to the host.

Optional - running with config.json

You can also provide the runtime parameters to Portworx via a configuration file called config.json. When this is present, you do not need to pass the runtime parameters via the command line. This may be useful if you are using tools like Chef or Puppet to provision your host machines.

  1. Download the sample config.json file: https://raw.githubusercontent.com/portworx/px-dev/master/conf/config.json

  2. Create a directory for the configuration file.

    sudo mkdir -p /etc/pwx
  3. Move the configuration file to that directory. This directory later gets passed in on the Docker command line.

    sudo cp -p config.json /etc/pwx
  4. Edit the config.json to include the following:

    • clusterid: This string identifies your cluster and must be unique within your etcd key/value space.
    • kvdb: This is the etcd connection string for your etcd key/value store.
    • devices: These are the storage devices that will be pooled from the prior step.

Example config.json:

   {
      "clusterid": "make this unique in your k/v store",
      "dataiface": "bond0",
      "kvdb": [
          "etcd:https://[username]:[password]@[string].dblayer.com:[port]"
        ],
      "mgtiface": "bond0",
      “loggingurl”: “http://dummy:80“,
      "storage": {
        "devices": [
          "/dev/xvdb",
          "/dev/xvdc"
        ]
      }
    }

Important: If you are using Compose.IO and the kvdb string ends with [port]/v2/keys, omit the /v2/keys. Before running the container, make sure you have saved off any data on the storage devices specified in the configuration.

Please also ensure “loggingurl:” is specified in config.json. It should either point to a valid lighthouse install endpoint or a dummy endpoint as shown above. This will enable all the stats to be published to monitoring frameworks like Prometheus:

You can now start the Portworx container with the following command:

if `uname -r | grep -i coreos > /dev/null`; \
then HDRS="/lib/modules"; \
else HDRS="/usr/src"; fi
sudo docker run --restart=always --name px -d --net=host       \
                 --privileged=true                             \
                 -v /run/docker/plugins:/run/docker/plugins    \
                 -v /var/lib/osd:/var/lib/osd:shared           \
                 -v /dev:/dev                                  \
                 -v /etc/pwx:/etc/pwx                          \
                 -v /opt/pwx/bin:/export_bin                   \
                 -v /var/run/docker.sock:/var/run/docker.sock  \
                 -v /var/cores:/var/cores                      \
                 -v ${HDRS}:${HDRS}                            \
                portworx/px-dev

At this point, Portworx should be running on your system. To verify, type:

docker ps

Authenticated etcd and consul

To use etcd with authentication and a cafile, use this in your config.json:

"kvdb": [
   "etcd:https://<ip1>:<port>",
   "etcd:https://<ip2>:<port>"
 ],
 "cafile": "/etc/pwx/pwx-ca.crt",
 "certfile": "/etc/pwx/pwx-user-cert.crt",
 "certkey": "/etc/pwx/pwx-user-key.key",

To use consul with an acltoken, use this in your config.json:

"kvdb": [
   "consul:http://<ip1>:<port>",
   "consul:http://<ip2>:<port>"
 ],
 "acltoken": "<token>",

Alternatively, you could specify and explicit username and password as follows:

 "username": "root",
 "password": "xxx",
 "cafile": "/etc/pwx/cafile",

Access the pxctl CLI

Once Portworx is running, you can create and delete storage volumes through the Docker volume commands or the pxctl command line tool. With pxctl, you can also inspect volumes, the volume relationships with containers, and nodes.

To view all pxctl options, run:

pxctl help

For more information on using pxctl, see the CLI Reference.

Now, you have successfully setup Portworx on your first server. To increase capacity and enable high availability, repeat the same steps on each of the remaining two servers.

To view the cluster status, run:

pxctl status

Adding Nodes

To add nodes in order to increase capacity and enable high availability, simply repeat these steps on other servers. As long as Portworx is started with the same cluster ID, they will form a cluster.

Application Examples

Then, to continue with other examples of running stateful applications and databases with Docker and Portworx, see this link.



Last edited: Wednesday, Feb 24, 2021