Starting with Docker v1.13, the Docker has introduced a “managed plugin system”, together with a new “v2 plugin architecture”. Please note that the “legacy plugin system” (Docker v1.12 plugins) is still fully supported in the newer Docker versions.

To install and configure Portworx v2 Docker Plugin, please use the steps below.

Install PX plugin

Before installing the plugin, please:

  1. Create the following directories on the host system:

    $ sudo mkdir -p /etc/pwx /opt/pwx/bin /var/lib/osd /var/cores
    • these directories are no longer created automatically via v2 Docker plugin, but will be required so that PX-plugin can export pxctl CLI onto the host, share configuration files, etc.
  2. Make sure you have your key-value database ready (ie. preinstall etcd). For etcd installation instructions refer this doc.

  3. Ensure host system has some extra disk-storage (ie. /dev/sdc disk).

To install Portworx as V2 Docker plugin, please run:

$ sudo docker plugin install portworx/px:latest --alias pxd \
  opts="-k etcd://<Domain name or IP address of your etcd server> -c <Your unique cluster ID) -s <path to storage device>"

Plugin "portworx/px:latest" is requesting the following privileges:
 - network: [host]
 - mount: [/dev]
 - mount: [/etc/pwx]
 - mount: [/var/lib/osd]
 - mount: [/opt/pwx/bin]
 - mount: [/var/run/docker.sock]
 - mount: [/lib/modules]
 - mount: [/usr/src]
 - mount: [/var/cores]
 - allow-all-devices: [true]
Do you grant the above permissions? [y/N] y

You will need to grant the permissions above for the plugin to be installed.

The required permissions are explained below:

    > Sets PX to be a privileged plugin. Required to export block device and for other functions.

 - network: [host]
    > Sets communication to be on the host IP address over ports 9001-9003. Future versions will support separate IP addressing for PX.

 - mount: [/dev]
 - allow-all-devices: [true]
    > Allows PX to access all host devices. Note that PX uses only devices/drives specified via `-s /dev/xxx` in opts or config.json.

 - mount: [/etc/pwx]
    > the configuration files location.

 - mount: [/var/run/docker.sock]
    > Used by Docker to export volume container mappings.

 - mount: [/var/lib/osd]
    > Location of the exported container mounts. This must be a shared mount.

 - mount: [/opt/pwx/bin]
    > Exports pxctl, the PX command line tool, from the plugin to the host.

The description of all of the arguments one can provide to the plugin via opts="..." install parameter:

-c                        [REQUIRED] Specifies the cluster ID that this PX instance is to join
-k                        [REQUIRED] Points to your key value database, such as an etcd cluster or a consul cluster
-s                        [REQUIRED unless -a is used] Specifies the various drives that PX should use for storing the data
-e key=value              [OPTIONAL] Specify extra environment variables
-v <dir:dir[:shared,ro]>  [OPTIONAL] Specify extra mounts
-d <ethX>                 [OPTIONAL] Specify the data network interface
-m <ethX>                 [OPTIONAL] Specify the management network interface
-z                        [OPTIONAL] Instructs PX to run in zero storage mode
-f                        [OPTIONAL] Instructs PX to use an unmounted drive even if it has a filesystem on it
-a                        [OPTIONAL] Instructs PX to use any available, unused and unmounted drives
-A                        [OPTIONAL] Instructs PX to use any available, unused and unmounted drives or partitions
-j                        [OPTIONAL] Specifies a journal device for PX
-x <swarm|kubernetes>     [OPTIONAL] Specify scheduler being used in the environment
-t <token>                [OPTIONAL] Portworx lighthouse token for cluster
  • additional PX-OCI -specific options:
-oci <dir>                [OPTIONAL] Specify OCI directory (default: /opt/pwx/oci)
-sysd <file>              [OPTIONAL] Specify SystemD service file (default: /etc/systemd/system/portworx.service)
-userpwd <user:passwd>    [OPTIONAL] Username and password for ETCD authentication
-ca <file>                [OPTIONAL] Specify location of CA file for ETCD authentication
-cert <file>              [OPTIONAL] Specify location of certificate for ETCD authentication
-key <file>               [OPTIONAL] Specify location of certificate key for ETCD authentication
-acltoken <token>         [OPTIONAL] ACL token value used for Consul authentication
-secret_type <aws|kvdb|vault>   [OPTIONAL] Specify the secret type to be used by Portworx for cloudsnap and encryption features.
-cluster_secret_key <id>        [OPTIONAL] Specify the cluster wide secret key to be used when using AWS KMS or Vault for volume encryption.
PX_HTTP_PROXY         [OPTIONAL] If running behind an HTTP proxy, set the PX_HTTP_PROXY variables to your HTTP proxy.
PX_HTTPS_PROXY        [OPTIONAL] If running behind an HTTPS proxy, set the PX_HTTPS_PROXY variables to your HTTPS proxy.
PX_ENABLE_NFS         [OPTIONAL] Enable the PX NFS daemon. Set PX_ENABLE_NFS=true.

NOTE: Setting environment variables can be done using the -e option, during PX-OCI or PX Docker Container command line install (e.g. add -e VAR=VALUE option).

# Example PX-OCI config with extra "PX_ENABLE_CACHE_FLUSH" environment variable
$ sudo /opt/pwx/bin/px-runc install -e PX_ENABLE_CACHE_FLUSH=yes \
    -c MY_CLUSTER_ID -k etcd:// -s /dev/xvdb

Staged install/startup of v2 Portworx plugin

Sometimes it will be more appropriate to install and start the v2 Portworx plugin “in stages”. This can be achieved using the following steps:

  • step 1: Download the PX-Plugin, but do not immediately enable it:
sudo docker plugin install --grant-all-permissions --disable --alias pxd portworx/px:latest
sudo docker plugin ls
ID                  NAME                DESCRIPTION                         ENABLED
9c6d7647ec0b        pxd:latest          Portworx Data Services for Docker   false
  • step 2: Configure PX-Plugin:
sudo docker plugin set pxd \
   opts='-k etcd:// -c MY_CLUSTER_ID -s /dev/sdc -d enp0s8 -m enp0s8'
  • step 3: Stop the old (v1) PX-Plugin container (if any), and start the v2 PX-plugin:
sudo docker stop px-enterprise
sudo docker update --restart=no px-enterprise

sudo docker plugin enable pxd

Migrating Portworx Container to Portworx v2 Docker plugin

Note that one cannot run the PX-Container and PX-Plugin at the same time. If you have previously installed Portworx as a Docker container (the “legacy plugin system”, or v1 plugin), please first stop the PX-Container and disable the automatic startup, like so:

sudo docker stop px-enterprise
sudo docker update --restart=no px-enterprise

Please make sure to install the Portworx v2 plugin with --alias pxd option during plugin installation. This option will enable Docker to find the registered PX-Volumes under new Portworx v2 plugin management, and transparently update the existing Docker containers/applications that use the PX-Volumes:

  • ie. PX volume used by MySQL before the v1->v2 Plugin update:
# docker inspect --format '{{json .Mounts}}' pxMySQL 
    "Type": "volume",
    "Name": "pxMysqlData1",
    "Source": "/var/lib/osd/mounts/pxMysqlData1",
    "Destination": "/var/lib/mysql",
    "Driver": "pxd",
    "Mode": "",
    "RW": true,
    "Propagation": ""
  • … and after the v1->v2 Plugin update (note: Driver and Source got updated):
# docker inspect --format '{{json .Mounts}}' pxMySQL 
    "Type": "volume",
    "Name": "pxMysqlData1",
    "Source": "/var/lib/docker/plugins/4e0c09be71b29399...aefae6/rootfs/var/lib/osd/mounts/pxMysqlData1",
    "Destination": "/var/lib/mysql",
    "Driver": "pxd:latest",
    "Mode": "",
    "RW": true,
    "Propagation": ""

Migrating Portworx v2 Docker plugin to PX-OCI or Portworx Container

If you require a migration from Portworx v2 Docker plugin to PX-OCI or plain Portworx Container - please contact support.

Optional - running with a custom config.json

You can also provide the runtime parameters to PX via a configuration file called config.json. When this is present, you do not need to pass the runtime parameters via opts argument. This maybe useful if you are using tools like chef or puppet to provision your host machines.

  1. Download the sample config.json file:
  2. Create a directory for the configuration file.

    # sudo mkdir -p /etc/pwx
  3. Move the configuration file to that directory. This directory later gets passed in on the Docker command line.

    # sudo cp -p config.json /etc/pwx
  4. Edit the config.json to include the following:
    • clusterid: This string identifies your cluster and must be unique within your etcd key/value space.
    • kvdb: This is the etcd connection string for your etcd key/value store.
    • devices: These are the storage devices that will be pooled from the prior step.

Example config.json:

  "clusterid": "make this unique in your k/v store",
  "kvdb": [
  "storage": {
    "devices": [

At this point, Portworx should be running on your system. To verify, run docker plugin ls.

Authenticated etcd and consul

To use etcd with authentication and a cafile, use this in your config.json:

  "kvdb": [
  "username": "root",
  "password": "xxx",
  "cafile": "/etc/pwx/cafile",

To use consul with authentication and a cafile, use this in your config.json:

  "kvdb": [
  "username": "root",
  "password": "xxx",
  "cafile": "/etc/pwx/cafile",

Access the pxctl CLI

After Portworx V2 plugin is running, you can create, delete & manage storage volumes through the Docker volume commands or via the pxctl command line tool, as you usually would.

A useful pxctl command is pxctl status. The following sample output of pxctl status shows that the global capacity for Docker containers is 128 GB.

# /opt/pwx/bin/pxctl status
Status: PX is operational
Node ID: 0a0f1f22-374c-4082-8040-5528686b42be
 	Local Storage Pool: 2 pools
	0	LOW		64 GiB	1.1 GiB	Online	b	us-east-1
	1	LOW		128 GiB	1.1 GiB	Online	b	us-east-1
	Local Storage Devices: 2 devices
	Device	Path		Media Type		Size		Last-Scan
	0:1	/dev/xvdf	STORAGE_MEDIUM_SSD	64 GiB		10 Dec 16 20:07 UTC
	1:1	/dev/xvdi	STORAGE_MEDIUM_SSD	128 GiB		10 Dec 16 20:07 UTC
	total			-			192 GiB
Cluster Summary
	Cluster ID: 55f8a8c6-3883-4797-8c34-0cfe783d9890
	IP		ID					Used	Capacity	Status	0a0f1f22-374c-4082-8040-5528686b42be	2.2 GiB	192 GiB		Online (This node)
Global Storage Pool
	Total Used    	:  2.2 GiB
	Total Capacity	:  192 GiB


  • Q: My PX-Plugin won’t start! The docker plugin ls shows Enabled=false even after I ran docker plugin enable pxd command. How can I fix it?
    • A: Please run journalctl -b -u docker to get the PX-Plugin log, and:
      • if you spot "bind: address already in use" error messages, please make sure you are not running both PX-Container and PX-Plugin at the same time (ie. check “docker ps” and “docker plugin ls”).
      • If you find "PX upgrade in progress. Requires reboot to complete." error message in the log, disable the PX-Container and reboot the host system.
      • NOTE that one can disable the PX-Container by running:
        docker stop px-enterprise; docker update --restart=no px-enterprise
  • Q: Docker apps cannot find the PX-Volumes after v1->v2 upgrade. How do I fix this?
    • A1: Make sure you have not omitted the --alias pxd option during the plugin installation (ie. command docker plugin inspect pxd should work). Reinstall plugin otherwise.
    • A2: Use umount and pxctl host detach commands to manually detach the PX-Volume, restart Docker service and the Docker apps that are using the PX-Volumes (or, just reboot the host).
  • Q: The docker volume ls and inspect commands failing when run on PX-Volumes after v1->v2 upgrade.
    • A: The PX-Volumes were likely in use (mounted) during the v1->v2 upgrade. Please restart Docker service to fix this.
  • Q: Docker startup is slow, logs show Docker is trying to access /run/docker/plugins/pxd.sock file.
    • A: The /run/docker/plugins/pxd.sock file should have been removed when the PX-Container services have been stopped. If by any chance this file still exists on the host, please remove it manually.
  • Q: Are you getting a No such file or directory message when you use SELinux?
    • A1: Portworx has a solution to resolve the issue SELinux.